Machine learning principles
Page 1 of 22
Introduction to the principles
The use of artificial intelligence (AI) and machine learning (ML) systems have the potential to bring many benefits to society. However, for the opportunities be fully realised, they must be developed, deployed and operated in a secure and responsible way.
AI and ML systems are subject to novel security vulnerabilities that need to be considered alongside standard cyber security threats. When the pace of development is high – as is the case with ML and AI – security can often be a secondary consideration. Designing security in from the outset is our best route to cyber resilience, which means security must be a core requirement, not just in the development phase, but throughout the life cycle of the ML system.
Without properly understanding and mitigating these vulnerabilities, system designers can't assure stakeholders, and ultimately users, that an ML system is safe and secure.
About these principles
These principles are aimed at anyone developing, deploying or operating a system with a machine learning (ML) component. They are not a comprehensive assurance framework to grade a system or workflow, and do not provide a checklist. Instead, they help developers, engineers, decision makers and risk owners make informed decisions about the design, development, deployment and operation of their ML systems.
The principles help underpin the NCSC’s Guidelines for secure AI system development, which are aimed at providers of AI systems, whether created from scratch or built on top of tools and services provided by others.
These principles have been developed by considering a generic ML workflow, and focus on pragmatic ways to address ML vulnerabilities. This allows the principles to cover a wide range of systems that are agnostic to data type, model algorithm or deployment environment.
Note:
The principles do not specifically cover reinforcement learning (RL) due to the fundamental differences in the way that RL applications are developed. The principles may provide a foundation when undertaking RL development, however additional security considerations (outside the scope of this document) are likely required.
Applying the principles
These principles should be considered in addition to established cyber security, risk management, and incident response best practice. In particular, we urge providers to follow the ‘secure by design’ principles developed by the US Cybersecurity and Infrastructure Security Agency (CISA), the UK National Cyber Security Centre (NCSC), and all our international partners. The principles prioritise:
-
taking ownership of security outcomes for customers
-
embracing radical transparency and accountability
-
building organisational structure and leadership so ‘secure by design’ is a top business priority
The UK government's Code of Practice for Software Vendors, designed to ensure that security is fundamental to developing and distributing products and services, will also help in this respect.
Note that not all of the principles will be directly applicable to all organisations. The level of sophistication and the methods of attack will vary depending on the adversary targeting the ML system, so the principles should be considered alongside your organisation's use cases and threat profile.
Each principle addresses the following:
- 1
What are the goals of the principle?
- 2
Why is it important?
- 3
How could this principle be implemented?
Terminology
A few quick points on terminology before we start.
Artificial intelligence (AI) describes computer systems which can perform tasks usually requiring human intelligence. This could include visual perception, speech recognition or translation between languages.
Machine learning (ML) is a type of AI by which computers find patterns in data or solve problems automatically without having to be explicitly programmed. Almost all AI in current use is built using ML techniques.
Large language models (LLMs) use algorithms trained on a huge amount of data, turning relationships between pieces of data into probabilities to predict sequences of text (or increasingly other content) in response to user prompts.
Continual learning (CL) comprises methods of adjusting or continuing to train a model once it is in deployment, based on operational interactions and feedback.
Note:
For ease of reference, we've collated all the external references used in these principles into the 'Further reading' section.
