Intelligent security tools
Assessing intelligent tools for cyber security
Pages
Page 3 of 6
Establishing the need
Intelligent tools are often sold as the solution to many of the problems and challenges we face in cyber security. They can perform with accuracy levels far beyond simple rule based tools and can automate processes on a scale that would be impossible for people to achieve.
However, there are many reasons why an AI-enabled tool may not be right for the issues you face.
The principles detailed in this section cover some of the key points you should consider when choosing an intelligent tool.
Comparing intelligent tools
When searching for security products you will naturally want to compare the relative merits of the solutions available to you.
In mature tool classes, such as antivirus, it's possible to compare like for like when purchasing.
However, this is not always possible with new intelligent tools, since:
- Their purpose may be to solve problems where there is no existing solution
- They might share the purpose of a well established tool, but aim to do this in a new 'intelligent' way
- They may be the only product on the market that has this purpose
Responsibility
We cannot advise on the utility of every possible function promised by new tools. However, it should be noted that tools will generally have an impact on your system in one of two ways. Either they will give people information to inform their decisions and actions, or the action will be fully automated. In both cases there is a question about who is responsible for the actions taken.
On the face of it, a tool which simply supplies the user with information leaves overall control in the hands of the person acting on that information. However, there is no guarantee that the operator will detect mistakes and may just become accustomed to following the tool's recommendations. In this scenario, effective control and responsibility have been given over to the tool.
Principles
These principles will help you identify tools which have functionality suited to your problem and way of working. They also highlight the risks of handing over tasks to a tool that may sometimes be unpredictable.
Before considering an intelligent tool, you need to think about the kinds of security problems that you face as an organisation.
It's important to be sure you have a real problem which is not being addressed by other means. You should explore whether augmenting your current approaches would be a sufficient remedy. This entails a clear understanding of the risk which your problem poses to your organisation.
Maintaining your current solution in tandem with any intelligent systems will not only ensure you have a fallback you understand, it will also give you a baseline against which to measure the outcomes produced by the AI based approach.
You should also think about how a solution might impact your business functionality, ensuring it will not impede core operations.
Once you have an intelligent tool in use, you will want to establish whether it is performing it's intended task. Understanding your issue, and your current mitigations, before implementing a new tool, will help you establish the value of the tool once it's up and running. You should also take the time to consider if it is better to use the new tool to augment your current processes, rather than replacing them entirely.
Establishing utility
- How does the decision, task, or automated action of the tool, contribute to the security of your organisation? What is the impact of this being incorrect?
- If the tool provides information, who will be using this information? Will this information help them make decisions or perform their task?
- How would the tool improve on the information already available? How will it improve the actions already taken?
- What process is currently used to make the decision, or undertake the task, or action? Is the tool going to augment this process, or replace it?
Sometimes an intelligent tool will not be the right solution for your problem.
There are some problems that AI will struggle with. And, even if a problem is solvable by AI, you may still find that it's quicker and cheaper for a task to be undertaken by a person.
There are also some problems that cannot be fully automated for legal reasons. For example, GDPR puts restrictions on automated decision making.
Complexity
AI is much better than simple rule-based automation at handling complex tasks. In fact, this is the primary purpose of many intelligent tools.
However, AI will also introduce additional complexity into your system, reducing your ability to understand the behaviour of your system, making it potentially more unpredictable.
This unpredictability could introduce new vulnerabilities to your systems, and these could potentially be exploited by attackers.
Black box syndrome
You may face scenarios where you need to understand or justify how a security-related decision was made. This can be extremely difficult with intelligent tools, since the algorithms used in many of them are 'black boxes'. This is not just a proprietary problem - we simply do not understand enough about how these algorithms reach their decisions to give a full elaboration of 'why' a particular choice was made.
As the complexity of the tools increases, there is a risk that it will become more difficult to understand how decisions are reached. Ongoing research into Explainable AI may be able to provide more insight in the future, but for now, it remains one of the limitations of the technology.
If you feel that understanding how a decision was made is a requirement of your problem, then you should look for a tool that offers this ability, or you should use an alternative method to reach that decision.
Establishing appropriateness
- Is this a task that should be automated by AI? Are there any legal considerations?
- Is there any advantage in using AI to automate this process, or is it best done by a person?
- Do you need to understand how decisions are reached? Is this information available in the tool?
Handing over actions to an intelligent tool means key decisions may be taken without a human having the opportunity to intervene.
Even if, in practice, a person is ultimately responsible for carrying out the action suggested by the tool, they may not be given sufficient information to understand when they should challenge the decision, or might not feel supported enough to do so.
This could be particularly difficult where decisions are being made in real time and the operator needs to respond quickly.
When to intervene
You need to understand how the intelligent tool will impact decisions made within your organisation, and who may be expected to intervene, to prevent mistakes.
Ultimately, you should know who is accountable for decisions and actions made or influenced by the intelligent tool. This person, or persons, should have the opportunity to act to change the decision made by the tool, and feel enabled to do so.
Unfortunately, some tools provide little or no opportunity to intervene. Whenever this is the case, the associated risk must be recognised by a suitable risk owner, when choosing the tool. Where intervention isn't part of the design, the only way to change a decision is to bypass the intelligent tool entirely. You should consider the alternative ways of making those decisions independent of the tool.
Sensitive information
You should think carefully about the information produced by the tool. It could be personal or sensitive, or require actions beyond the capabilities of the person receiving the information.
For example, tools observing the behaviour of your staff may inadvertently reveal information about their personal lives, and tools scanning for vulnerabilities may report far more than can be acted upon.
Those receiving this information need to be supported so that the information is properly handled, and they don't inadvertently end up carrying the risk of holding that information.
Understanding the risks
- Consider the potential consequences of handing over control of an action to the tool. What could happen if it doesn't perform correctly?
- What opportunities do the operators have to prevent or change the action taken by the tool? What processes are in place to ensure errors don't have catastrophic impacts?
- Who will need to account for the decisions made by the tool? Do they have the information needed to understand how the tool reached its decision? Do they have the ability to ensure mistakes are not repeated?
- Who is responsible for carrying out actions suggested by the tool? Is that person able to take a different action if needed, and will they be supported to make that decision?
- What are the handling requirements for the information produced by the tool? What processes will need to be put in place to ensure it is handled correctly?
- Who is responsible for the information discovered by the tool? Are they sufficiently trained to know how to handle and act on the information?
