Penetration testing, or security testing in general, is the targeted search for vulnerabilities in order to assess the security level of an IT environment and make informed decisions accordingly. What sounds so simple and obvious is often much more complex in practice and frequently not as clearly defined as it may seem.

Penetration testing is an established and well-known practice in the field of information security. Penetration tests are an integral part of many security programs and various industry standards and supervisory authorities stipulate the regular performance of penetration tests. This is positive because it demonstrates that this very effective means of increasing the security level is widely adopted. At the same time there is a danger that because of the habituation effect penetration tests may not be viewed with the necessary care or may be seen purely as a compliance task.

This article looks at a few simple but essential points that contribute significantly to the success and benefits of security tests. In the field of offensive information security, many terms such as penetration testing, ethical hacking, red teaming and others are used. For the purpose of this article, the term security testing is used generically and refers to the entire spectrum of offensive security testing.

Why Security Testing?

Simply put: To determine the actual security level of an IT system, an IT environment or even an entire organization. The aim is to find out as realistically as possible what can happen in the event of real attacks, how far an attacker could get and what the resulting impact could be. In other words, the existing security measures are subjected to a reality check, which identifies weaknesses and potential gaps that could be exploited by real attackers.

With the insights obtained through testing, risks can be realistically assessed and decisions about necessary countermeasures are supported, enabling an organization to optimize its security level. Regular security testing can make a significant contribution to reducing the risk of security incidents and data leaks.

One might ask why security testing is still needed today, given the huge number of security products and services available. The answer is simple: The effectiveness of security measures cannot be adequately assessed without testing them. In addition, IT is particularly subject to constant change, therefore security measures need to be constantly adapted and tested. The idiom “Trust is good, control is better!” is particularly relevant in the field of information security.

Hypothesis and Experiment

The classic approach of the scientific method with the definitions of hypothesis and experiment can be applied very well to security testing. This is valuable because it allows the prerequisites and success factors to emerge more clearly.

A hypothesis is an unproven statement or an assumption of facts or laws. A hypothesis is usually derived from a theory, for example in our context: Secure development processes mean secure software, best practices work, security products provide sufficient protection, compliance means security and so on. Corresponding hypotheses can be: Sufficient security measures have been implemented, therefore the environment is secure, the procured product is secure, the organization is secure because it is compliant, the system is isolated in such a way that it is inaccessible to attackers.

An experiment is a methodically designed examination to obtain information empirically. This is precisely the purpose of a penetration test. The value of an experiment is measured by the correctness, usability, and usefulness of the obtained information. This information answers questions, that is, it confirms or invalidates the hypothesis made. Therefore it becomes immediately clear that everything starts with the right questions.

Suitable questions could be:

• Is an attacker able to gain access to sensitive data or systems?
• Are known vulnerabilities present in the technologies used?
• Are the necessary controls implemented to establish an appropriate level of security?
• Are the mandated security requirements met?
• Was the concept correctly implemented and does it work in practice?
• Can an existing concern be refuted or confirmed?
• Do the security relevant processes work correctly?
• How difficult or easy is it for an attacker to succeed?
• Are the detection and response mechanisms sufficiently effective?

When planning penetration tests, this frequently receives too little consideration. It is often not sufficiently clear which questions a security test should answer and what the answers are to be used for. The preparation and planning phase is therefore particularly important in security testing and must not be neglected. It can be challenging for a customer to know what options are available, what restrictions exist and what points need to be considered. If these aspects are paid too little attention to or are agreed on too vaguely, there is a risk that money will be spent on a test that is not useful enough or that resources cannot be used effectively, and the test will have to be repeated. An experienced service provider provides neutral and objective support and guidance during this phase and ensures that the questions to be answered by the test and the means to be used are clarified before the test assignment is placed.

After all, a penetration test is only a means to an end, namely, to gain information that can be used to make decisions. Usually, the findings are interpreted by the service provider as far as possible and are assessed with severity levels and suitable measures are recommended to limit the risks associated with the findings. It is then up to the customer to decide what the findings mean for them and how to deal with them.

The importance of being clear about the questions to be answered is emphasized once again. In this context, a few of the most common approaches in security testing are compared below to show which questions they can answer.

Comparison of three common security testing approaches

We’re explaining different security testing approaches down below in more detail, including penetration testing and red team testing, as well as the main differences between penetration testing and a bug bounty.

Penetration Testing and Red Team Testing

Penetration tests are usually executed out to examine the security of a test object with a defined scope from the perspective of a real attacker. It is tested whether the networks, platforms, applications, systems, etc. in the scope have vulnerabilities that can be exploited by an attacker. Care is taken to carry out as complete an examination of the test object as possible within the agreed scope. For example, for testing web applications the OWASP Testing Guide is usually used, which includes all topics relevant to the security of web applications. The aim is to achieve comprehensive test coverage to avoid partial statements with limited benefit. Penetration tests are used for the comprehensive investigation of a given scope and are not normally designed to evade detection or to test the detection capabilities of a SOC.

Red team tests attempt to achieve a defined goal from an agreed starting point, such as gaining access to sensitive data or systems. In contrast to a penetration test, however, the focus here is on simulating an attacker who uses all relevant means and methods to achieve the defined goal while remaining as undetected as possible. The aim is therefore not to identify as many weak points as possible in a given scope, but to find weak points that will help to achieve the defined objective. Accordingly, the SOC or the Blue Team is generally not pre-informed during Red Team tests, which also allows statements to be made about the detection and response capabilities regarding advanced attacks. Red team tests are useful if the examined organization already has a certain level of maturity in information security.

The methods and tools used in penetration tests and red team tests are in many cases similar or even the same. Red team tests focus on the resilience of an organization to real, advanced attacks, while penetration tests generally relate to individual systems or areas.

Penetration Testing and Bug Bounty

Bug bounty programs have become increasingly popular in recent years. If set up correctly, important insights can be obtained with a defined maximum expenditure and in a comparatively short time. The most important differences between penetration tests and bug bounties are compared in the following table.

Bug bounty programs and penetration tests both aim to identify vulnerabilities, but differ in their engagement models, incentives, and areas of focus. Bug bounty programs leverage the expertise of a large number of testers, offer financial rewards for reporting vulnerabilities, and can provide quick, but perhaps not always comprehensive, results. Penetration testing takes a holistic approach, offers easier interaction with testers, and is conducted by a small number of dedicated experts.

Conclusion

A security test is an experiment that provides information and thereby answers questions. It is therefore very important for the customer and for the service provider to be clear about which questions are to be answered by the test and what the results are to be used for before starting a security test. This is a general statement which applies equally to the various types of security tests. An experienced service provider provides neutral and objective support and advice during this phase and ensures that it is clear which questions the test is intended to answer, and which methods are to be used.

About the Author

Tomaso Vasella has a Master in Organic Chemistry at ETH Zürich. He is working in the cybersecurity field since 1999 and worked as a consultant, engineer, auditor and business developer. (ORCID 0000-0002-0216-1268)

Links

• https://owasp.org/www-project-web-security-testing-guide/
• https://plato.stanford.edu/entries/scientific-method
• https://www.finma.ch/en/~/media/finma/dokumente/dokumentencenter/myfinma/rundschreiben/finma-rs-2023-01-20221207.pdf
• https://www.pcisecuritystandards.org/