10 questions to ask when using AI models to find vulnerabilities

Hopefully your answer is ‘to improve security within my organisation’. However it’s important to remember that just finding vulnerabilities does nothing to improve your security. You could even make your security worse.

The most important tool to improve your organisation’s security is the application of fundamental cyber security hygiene. A vulnerability on a system where a patch for it exists (but has not yet been deployed) or unauthorised access are much more likely to affect security than the exploitation of zero-days. So ask yourself, is your organisation carrying out best practice? Do you understand what software you rely on. What IT is in your estate?

You need to be able to manage vulnerabilities well, whether found using AI or not, especially as the number of reported vulnerabilities increases. You want to know how to receive, prioritise and fix issues, without teams being overwhelmed. Make sure you consider fixing the cause of the vulnerability too. The NCSC’s Vulnerability management guidance is a useful resource for those looking to set up and improve their process.

A system or product might have a lot of vulnerabilities, but the ones that attackers can exploit are the ones that need prioritising. Some vulnerabilities you might fix immediately, others might indicate you need a major rewrite to parts of code, or that you need to remove that attack service from your environment. There were over 40,000 vulnerabilities assigned CVEs in 2025. The CISA KEV says only about 400 new vulnerabilities were tracked as exploited, and only around 40 of those were zero-days when initially exploited. This is why prioritised patching is so important.

Using AI isn’t risk free and there are many security implications to consider. These include:

• How could I leak information?
• How will I secure the infrastructure used?  
• Have I sandboxed my system so that it can only talk to the LLM and my code base?  
• Will I give access to my production environment?  
• What permissions have I given the LLM?
• How can I avoid spending all my money/time/people finding vulnerabilities, and have nothing left to fix them?
• Do I understand the terms and conditions, and data retention policies?
• How will I ensure the activity is legal?

Different models have different properties. You don’t necessarily need access to the latest model; you can start using any model as this will build your experience and gives you an idea of the model’s capabilities. Before using a hosted model, the NCSC recommend you consider:

• the physical location and legal jurisdictions which apply to the hosted models
• any laws in that location and jurisdictions related to vulnerabilities

When using AI to find vulnerabilities prioritise your external attack surface. You should also look for ways to verify the results, by using both AI and humans.

The NCSC's view is that keeping pace with frontier AI cyber developments will almost certainly be critical to cyber resilience for the decade to come. New models will come out, they will have different capabilities, and we expect these to improve. Therefore, you need to consider:

• How you are going to resource this long term?
• How will you respond to new models?
• How are you going to engage with your customers?
• How are you going to help customers if they are reluctant to install any updates?
• How are you going to respond to vulnerabilities found in devices/libraries/services you use?

AI is a tool that attackers and defenders will use. Organisations will benefit from combining the capabilities of AI models with staff who understand security. We believe that AI models accelerate the skills of cyber security staff; they don’t replace them.

It’s really important to understand the patching regime in your organisation, and think about how it’s going to change over the next few years. The first step for many organisations involves understanding the entirety of their estate, and identifying the critical products and services. Good asset management and dependency management are crucial.