Severities and Risks are not the same
An essential part of the report is the assessment of the detected vulnerabilities from a security perspective. But what is the meaning of these ratings and how is the recipient of the test results supposed to deal with these conclusions? This is a question that repeatedly leads to discussions and uncertainties, especially regarding the risk management process. These relationships and a pragmatic approach to handling the ratings of findings are discussed in this article.
The findings of security tests are typically rated using a scale with several levels as illustrated in the following example.
These severities are a measure for the quality and the effectiveness of the implemented security controls for each test case. They range from controls cannot be circumvented to controls can be circumvented directly and confidentiality, integrity and availability cannot be guaranteed. Thus, the rating of a weakness reflects its severity or in other words, is a measure of the adequacy of the existing security controls. These ratings refer to the conducted test cases only, they usually do not include additional contextual factors.
Many definitions of the term risk exist. Two selected ones that are suitable in the context of cyber security are quoted below:
The potential that a given threat will exploit vulnerabilities of an asset or group of assets and thereby cause harm to the organization. It is measured in terms of a combination of the probability of occurrence of an event and its consequence.
A measure of the extent to which an entity is threatened by a potential circumstance or event, and typically a function of: (i) the adverse impacts that would arise if the circumstance or event occurs; and (ii) the likelihood of occurrence.
So, a risk is more than just a rating of a weakness. Risks consider the context of a weakness and the possible adverse effects. In addition to the severity of a weakness and the extent of possible negative consequences, the rating of risks also includes relevant threats, existing risk-reducing measures and the likelihood or probability that the vulnerability will actually be exploited. For the qualitative rating of risks (e.g. low, medium, high) based on the impact and the probability of occurrence, a corresponding risk matrix is usually used, as shown in the following example.
The severity of a weakness only changes if the test object itself is changed, for example by remediating the weakness by means of a bug fix or a configuration change. However, the context of a weakness, i.e. its environment is practically never constant over time. Therefore, a risk rating is a snapshot and can change over time, even if the severity of the underlying weakness doesn’t change.
An example: let’s assume that a penetration test identified a weakness in a network service and its exploitation easily allows for the unauthorized exfiltration of sensitive data from the affected system. Thus, the severity of this weakness would probably be rated das high or possibly even as emergency. However, the system is located in an internal, isolated network segment with strict access control and a very small number of users. Because of this, the probability that the weakness is actually exploited is much smaller than if the system were directly accessible from the Internet without additional protection. Therefore, the risk associated to this weakness would most likely not be rated as high but rather as medium or even as low because of the specific context. However, in case this system would be moved into a different, less protected network segment, the risk rating would increase correspondingly. This clearly shows that rating and managing risks is not a single, one-time action but must be carried out as part of a continuous risk management process.
The following table summarizes the most important differences between the terms severity and risk.
|Rating of a weakness of a test object without including its context||Assessment of extent and probability considering the relevant context (threats, impact, existing controls, etc.)|
|Can change due to changes of the test object||Can change due to changes of the test object or due to changes of the context|
|Provides information about the adequacy of the existing security controls||Provides information about the possible extent of the impact and the probability that such impact will occur|
|Can be rated by an external tester||Requires additional information about the context of the test object and therefore can usually not be assessed by an external tester|
As part of their risk management, organizations often also take cyber risks into account or are even obliged to do so by the law or by industry regulations. This often raises the question of how to deal with weaknesses and their ratings from a risk management perspective. In practice, the following pragmatic approach has worked well to compile a qualitative risk rating based on the severity of an underlying weakness:
The resulting risk is then looked up in the risk matrix. It is often worth to base the reflections about impact and probability on risk scenarios and to discuss these scenarios in the team. Risk scenarios are established by imagining the entire sequence of events of a certain attack and describing it with enough details to be able to explain the attack to others in an understandable and comprehensible way.
In practice, this process is often not as easy and straightforward to execute and lively discussions about probabilities and risk thresholds may arise. While impact ratings typically relate closely to severities, humans are inherently bad at estimating probabilities and our own intuition easily misleads us. In real life, we are dealing with frequencies, not probabilities, and it is well known that our perception tends to be unconsciously influenced (cognitive bias, see Logical Fallacies when Assessing Risks). Therefore, when managing cyber risks, it is all the more important to focus on the facts, to analyze the situation carefully and to be aware of the context under which the risks were rated. External experts can provide advice and explain the issues they identified in their tests. However, the actual risk assessment must be performed by the organization itself because in most cases, only the organization has sufficient context information.
The apparent discrepancy between the rating of a vulnerability and the rating of the corresponding risk that sometimes occurs can lead to discussions. Often, similar terms and colors are used to indicate severities and risks, for example red or high. It is possible that the severity of weakness is rated as high (red) while the associated risk is rated medium (yellow). This can trigger requests for explanations because supposedly the assessment of the security tester was altered during the risk management process while actually only the difference between severity and risk was overlooked. Again, this illustrates the need for concentrating on the facts and for ensuring transparency by using suitable methods for data presentation.
An external security test identifies weaknesses and rates them in terms of severity. The distinction between severities and risks is important and requires a careful examination of the relevant facts and the test cases. Rating risks based on identified weaknesses and their severities can only be conducted by considering the context of the weaknesses, a process that usually must be performed by the respective organization.
Finally, it must not be forgotten that both the results of security tests and the rating of the corresponding risks reflect the security state at the time of observation. Given today’s dynamic nature of cybersecurity, regular security checks and continuous risk management practice are therefore mandatory.
Our experts will get in contact with you!
Our experts will get in contact with you!
Further articles available here