Blog post

Rating hackers, rating defences

Created:  06 Sep 2018
Updated:  06 Sep 2018
Author:  Mat P
Part of:  Cyber attacks
Running track

Today we often struggle to articulate how difficult it is to compromise a given system in some way. Wouldn't it be good if we had a common vocabulary so that these issues could be discussed, knowing that you were being understood?


Lack of understanding

Many people find it difficult to differentiate between a vulnerability in their systems which is exploitable by anyone who downloads a free online tool, and one which is likely to require months of effort from a dedicated team of super-skilled attackers. Wouldn't it be useful for the NCSC (and the cyber security community more widely) if we could adopt a common language to describe both the sophistication required for a particular attack, and the sophistication shown by a given threat actor*.


The de-facto standard

At least one de-facto standard is emerging already: STIX (Structured Threat Information eXpression). This is a language developed for cyber threat intelligence sharing. The version 2.0 draft (in section 6.11) defines Threat Actor Sophistication in 7 levels, from None ('Can carry out random acts of disruption or destruction by running tools they do not understand'), all the way through Minimal, Intermediate, Advanced, Expert, and Innovator, to Strategic ('State actors who create vulnerabilities through an active program to influence commercial products and services during design, development or manufacturing, or with the ability to impact products while in the supply chain to enable exploitation of networks and systems of interest').


Why is a common vocabulary useful?

The argument boils down to precision and understanding. With a set of agreed terms we can have more meaningful discussions about security. This will enable us to more closely match our defensive efforts to expected threat levels. I've outlined eight situations which would clearly benefit from a common vocab. You might be able to think of more.

1. Risk appetite

Any organisation can use these terms to contribute to cyber security risk appetite. For example:

  • 'We want to ensure that customer data is secure from an Advanced level attack.'
  • 'We want to be able to defend our operational systems from Intermediate level attacks but need to be able to detect Advanced and recover from Expert level.'

2. Is it secure?

We can move away from the vagueness of questions like 'Is it secure?', and instead ask, 'Is it secure against attacks requiring an Intermediate level of capability?'

3.Setting priorities

The ability to give a clear picture of threats and defences, will aid dialog about priorities with senior stakeholders. For example, in government, should we direct our limited cyber security resources towards ensuring that all government departments cannot be compromised by mere Intermediate level attacks, or ensuring that a smaller number of government systems can defend against Expert level?

4. Statistics

Concrete definitions of capability levels mean we can estimate how many people worldwide fit into each category. We can use these figures to support a narrative with our customers and stakeholders. For example, 'Do you realise that any of the 1.6 million people globally that we estimate are of Intermediate capability could compromise your system?' (that figure is made up). This seems more likely to drive the point home than 'we found a vulnerability in your system'.

5. System comparison

We could use it to compare one organisation or system against another. 'We think that Hogwarts would stand up against Advanced attacks, but Mordor's HR system could be compromised by them.' This would also allow us to set national strategies about the ability for specific types of organisations to defend against particular levels of attack.

6. Threats change over time

We could use this common vocabulary to compare attacks and levels of capability over time. For example: Stuxnet at the time it was discovered = Innovator. Stuxnet today = Advanced. Stuxnet next year  = Intermediate?

7. Effective practice

We could create exercising scenarios which replicate attacks of a specific capability level.

8. Smart choices

We could start to generate statistics to enable us to make data-driven decisions in future:

  • Tag incidents (this one used Advanced capability)
  • Tag pen-test reports (report concludes that the system could be compromised by an Intermediate level attack or above)
  • Tag designs (this one should be suitable to defend against Advanced attacks as along as the system is well maintained)



Assigning levels to specific attackers is difficult, especially for the most capable. The level assigned to an attacker represents the maximum capability they are currently judged to have. Threat actors will often use the easiest route they can to achieve their objectives, and will avoid deploying their most valuable capabilities unless truly necessary. They may therefore use tactics, techniques and procedures that may be used by much lower capability actors, including exploiting commonly known vulnerabilities and using publicly available tools. Some attackers may be extremely capable in one area (for example attacks against financial infrastructure), but far less capable in another (attacks against power infrastructure).


Experience so far

For some time now, I've been trialling the use of this STIX scale with a range of the NCSC's critical national infrastructure (CNI) clients. So far, they've all chosen to adopt it in some way internally, enjoying many of the advantages outlined above. Sometimes the organisations have cut down the number of points on the scale to suit their situation, feeling seven was too granular for their needs. The downside to this is that we lose the ability to easily compare between organisations.


What next?

There have been similar scales described in the past, and used for a range of purposes. There's not much new stuff here in that sense. The real benefits that a common scale could bring to cyber security won't be felt until we have widespread adoption. Using the same scale to describe both the maximum capability of a given attacker, and the measures required to detect, defend or recover from such an attacker will lead to more meaningful and nuanced discussions about cyber security. 

I'd be very pleased to hear any thoughts. I'm particularly keen to hear of any prior work which has been done in this area. You may have an opinion on whether STIX is the right vocabulary to use. And, looking further ahead, once we choose a vocabulary, where are the most important places to start using it? As usual, feel free to use the comments below or contact us directly.


Mat P

CTO for Private Sector Critical National Infrastructure

*'Threat actor' is the cyber security term used to describe a person or group who would seek to do harm. For example, nation state, hacktivist, terrorist group, bored person with too much free-time and a grudge.



Andrew - 07 Sep 2018
I find it amusing that the NCSC seem to be slowly realising that a lot of the 'reforms' they've made to Information Assurance over the years are actually damaging, and now we see ideas being 'invented' by the NCSC which were actually in-place and working effectively, before the NCSC attempted to kill them off. This is a prime example - IS1/2 already covers the concept of attacker capability, and SIRO submissions written by a competent CLAS consultant would include precisely these sorts of numbers. Why you need to reinvent the IS1/2 approach (via STIX) is beyond me!
Mat P - 09 Oct 2018
Worth clarifying something here. The rationale behind the changes to IS1 and 2 were mainly about removing the mandation behind that set of standards. We were in an unhealthy situation where everyone had to do the same thing – one size fit none (or very few, at any rate). There are elements of IS1 and 2 which are perfectly decent, and can add value to security risk management. The problem came from mandating the whole thing to everyone within government. We’re now moving towards a situation where we have a broader toolbox of risk management techniques, which includes some parts that look similar to elements of IS1 and 2. But, and this is crucial, this toolbox includes techniques which were not included in IS1 and 2. This puts the onus on individual risk managers to select what they feel is the most appropriate risk analysis technique. We’ve started to outline these techniques in our most recent risk management guidance (, which we’ll be expanding soon.
To be clear, NCSC cannot tell you which technique you should apply in every situation – this depends too much on organisational context. So, on the subject of STIX, as the blog states, this is a shared language that has currency internationally, which makes it useful to us, and to some of our customers. What we’re not saying is that everyone must use it. Only use it if it makes sense to you.
Rod Widdowson - 07 Sep 2018
Fascinating insight.

Andrew - 10 Sep 2018
I really like this! This is great.
A suggestion for improvement;

Consider turning this into a matrix, with the other dimension being attacker type (eg, hacktivist, organized crime, nation-state, etc.). I think this is important because different attackers have different motivations.
For instance, a hacktivist’s goals may be to publicly shame your organization. Organized crime may sell or blackmail your info. And a nation state may keep their activities quiet.
Another benefit is that instead of saying, “we’re susceptible to hacktivists,” you can say how sophisticated the hacktivists are. Some may be None while others could be Innovative or even Strategic.

Please feel free to contact me if you’d like to discuss this more. Again, I really like your idea and agree that the industry needs standardized language. This would greatly help decrease the amount of fear mongering that so often accompanies news reports of “hacking.”
Matthew - 11 Sep 2018
Great article!
Nick Tegg - 03 Oct 2018
Does this blog post indicate that NCSC is supportive of STIX 2.0 and would welcome the use of the STIX vocabularies when we are engaged in IA activities?
Mat P - 12 Oct 2018
NCSC support the use of STIX 2.0 for its original purpose of enabling organisations to share Cyber Threat Intelligence with one another in a consistent and machine readable manner.
There may also be broader uses where the definitions and vocabulary it provides can provide some consistency, STIX isn't the only option here, but it is a well-recognised one.

Leave a comment

Was this blog post helpful?

We need your feedback to improve this content.

Yes No