Risk management

Risk information is any information that can influence a decision. Some organisations have a tendency to only accept certain types of information as legitimate risk information. Such limitations increase the chance of something important being missed.

Imagine that your organisation’s cyber security risk management approach can only deal with qualitative information (such as policy papers, incident reports, or assessments) that describe risk in terms of high, medium and low. Such an approach would miss the patterns and trends that could be spotted by including quantitative information (such as network flows, or numbers of security incidents). Drawing on a variety of information sources may reveal risks that would otherwise be missed.

It is rare for organisations to explicitly exclude certain types of risk information, but they often have an unspoken bias towards a given type. Security is sometimes claimed to be ‘unquantifiable’, or qualitative information is discounted because it’s one person's (subjective) opinion. Again, these biases can cause organisations to overlook valuable information when conducting cyber security risk assessments.

You're more likely to fall into this trap if your organisation adopts a single, standardised approach for every kind of cyber security risk assessment. This is more likely to occur when organisations focus on completing the process of risk management or compliance, rather than on the risk management activities which should flow from it. When organisations get into this ‘defensive’ pattern of risk management behaviour, this closing down of what counts as ‘legitimate’ risk information can be exacerbated.

How can you know if you're considering enough information sources? This is more of an art than a science, and the technique we're suggesting below uses a matrix that classifies information as qualitative or quantitative, and objective or subjective:

• qualitative information is about describing something in human language, such as written information presented in documents
• quantitative information is about things that can be measured in numbers
• objective information is verifiable and not subject to opinion (such as the number of laptops that your organisation holds, or the amount of money it would cost you to purchase a particular antivirus solution)
• subjective information is a matter of opinion (such as the judgement that a particular organisation is more at risk of DDoS attacks than of ransomware attacks)

Start by assigning each information type above to the corresponding location on the grid. You'll quickly be able to identify if there are any potential blind spots, as any empty quadrants will be immediately apparent. We've completed an example grid below, with different types of risk information assigned to each quadrant.

We are not suggesting that information from any of the four quadrants is ‘better’ than any other type. The purpose of this exercise is to visualise the spread of information sources that you use in your risk analysis, so that you can quickly identify any blind spots. It won't tell you exactly what you're missing, but it can shine a light on organisational biases towards a particular type of information.

So how might you go about doing this?

1. Start by going through all the various information sources that feed into your organisation's risk assessment process.
2. Place them into the grid above. If you're not sure where to start, go back to a decision that related to cyber security in your organisation. What information was used to inform that decision? If nothing was written down, go back and speak to the person who made the decision, and ask them what they used to decide on that security issue.
3. Examine the grid. What does it look like? Are you weighted towards one quadrant, or perhaps one half?
4. What other information could you have gathered to fill the gaps? How might that have changed the decision? Why are the gaps where they are? What blind spots might this cause in your approach to risk analysis?

This is by no means the only way of categorising risk information. There are other properties of risk information which may be just as useful. For example, it is also worth considering whether you are using a balance of information about the past, and information about how you anticipate the future will unfold, with some interpretation. The majority of approaches used today to assess and analyse cyber security risk use qualitative and subjective information. Those interested readers who want to learn more about using quantitative information can read our Introduction to cyber risk quantification.

By using the qualitative/quantitative, objective/subjective technique, the NCSC have recognised a common bias in many organisations, where the grid is heavily populated in the top-left and bottom-right quadrants, and empty in the other two. In such organisations, when assessing cyber risk, the terms 'objective' and 'quantitative' were taken to mean the same thing.

Our findings demonstrate that in these organisations, information that was inconsistent with this flawed assumption was ignored. For example, experts' subjective assessments of probability were discounted.