The NCSC research problem book
Page 5 of 21
CC3 - How do we create and adopt meaningful measures of cyber security?
How do we create and adopt meaningful measures of cyber security?
Governments around the world are publishing their cyber security strategies, starting new initiatives to improve the cyber security of their citizens and businesses, or to minimise the harm in the wake of incidents. But as an industry, we have no means of measuring success or comparing results. Is the UK National Cyber Strategy living up to its objective to make the UK “the safest place to work and do business online” – or is another country’s cyber security strategy somewhere else making more of a difference?
Whether focusing at country scale, or within an individual organisation, we can't understand which investments in cyber security will make the biggest difference without a meaningful set of metrics and measurements.
Strands or sub-problems
-
Modelling
This involves defining meaningful taxonomies and measures of vulnerability, threat, impact and risk that can be applied to a single organisation, an individual sector or a whole country. Would models like those used in healthcare measuring population health also be suitable for cyber security? And if so, what are the right indicators to be measuring?
-
Data science
This is important to define the methodologies for applying these models and running experiments to measure impact of cyber security interventions or policies. What are the ethical considerations and best approaches for running these experiments?
-
Socio-technical security
This incorporates the human element in measuring security, as well as making sure the measures in decision making are understandable and actionable.
-
Data and tooling
Collecting and analysing the data needed to make comparative assessments of sectors or countries over time is a crucial step. Is there a simple but meaningful way for individual organisations to measure their own cyber security by applying the research from the other strands?
Why this is important
“A couple of years ago, we started wrestling with the problem of whether the UK is getting more or less secure in cyber space. As my area of expertise is vulnerabilities, I’ve focused on trying to understand this aspect of the problem – what do we mean by vulnerability, and how can we meaningfully measure it at country scale? In exploring this problem, we’ve identified some parallels with how population health is measured at country scale, using a set of domains and indicators, and we're looking at how we could apply this to cyber security. We think our results so far are promising, but we'd be keen to see much more research happening here across the community and internationally.”
Rob T, NCSC Technical Director for Vulnerability Research