Guidelines for secure AI system development

Principles for the security of machine learning
The NCSC’s detailed guidance on developing, deploying or operating a system with an ML component.

Secure by Design - Shifting the Balance of Cybersecurity Risk: Principles and Approaches for Secure by Design Software
Co-authored by CISA, the NCSC and other agencies, this guidance describes how manufacturers of software systems, including AI, should take steps to factor security into the design stage of product development, and ship products that come secure out of the box.

AI Security Concerns in a Nutshell
Produced by the German Federal Office for Information Security (BSI), this document provides an introduction to possible attacks on machine learning systems and potential defences against those attacks.

Hiroshima Process International Guiding Principles for Organizations Developing Advanced AI Systems and Hiroshima Process International Code of Conduct for Organizations Developing Advanced AI Systems
These documents, produced as part of the G7 Hiroshima AI Process, provide guidance for organisations developing the most advanced AI systems, including the most advanced foundation models and generative AI systems with the aim of promoting safe, secure, and trustworthy AI worldwide.

AI Verify
Singapore’s AI Governance Testing Framework and Software toolkit that validates the performance of AI systems against a set of internationally recognised principles through standardised tests.

Multilayer Framework for Good Cybersecurity Practices for AI — ENISA (europa.eu)
A framework to guide National Competent Authorities and AI stakeholders on the steps they need to follow to secure their AI systems, operations and processes

ISO 5338: AI system life cycle processes (Under review)
A set of processes and associated concepts for describing the life cycle of AI systems based on machine learning and heuristic systems.

AI Cloud Service Compliance Criteria Catalogue (AIC4)
BSI’s AI Cloud Service Compliance Criteria Catalogue provides AI-specific criteria, which enable evaluation of the security of an AI service across its lifecycle.

NIST IR 8269 (Draft) A Taxonomy and Terminology of Adversarial Machine Learning
A set of processes and associated concepts for describing the life cycle of AI systems based on machine learning and heuristic systems.

MITRE ATLAS
A knowledge base of adversary tactics, techniques, and case studies for machine learning (ML) systems, modelled after and linked to MITRE ATT&CK framework.

An Overview of Catastrophic AI Risks (2023)
Produced by the Center for AI Safety, this document sets out areas of risk posed by AI.

Large Language Models: Opportunities and Risks for Industry and Authorities
Document produced by BSI for companies, authorities and developers who want to learn more about the opportunities and risks of developing, deploying and/or using LLMs.

Introducing Artificial Intelligence
Blog from the Australian Cyber Security Centre which provides approachable guidance on Artificial Intelligence and how to securely engage with it.

Open-source projects to help users security test AI models include:

• Adversarial Robustness Toolbox (IBM)
• CleverHans (University of Toronto)
• TextAttack (University of Virginia)
• Prompt Bench (Microsoft)
• Counterfit (Microsoft)
  
• AI Verify (Infocomm Media Development Authority, Singapore)

CISA’s Cybersecurity Performance Goals
A common set of protections that all critical infrastructure entities should implement to meaningfully reduce the likelihood and impact of known risks and adversary techniques.

NCSC CAF Framework
The Cyber Assessment Framework (CAF) provides guidance for organisations responsible for vitally important services and activities.

MITRE's Supply Chain Security Framework
A framework for evaluating suppliers and service providers within the supply chain.

NIST AI Risk Management Framework (AI RMF)
The AI RMF outlines how to manage socio-technical risks to individuals, organisations, and society uniquely associated with AI.

ISO 27001: Information security, cybersecurity and privacy protection
This standard provides organisations with guidance on the establishment, implementation and maintenance of an information security management system

ISO 31000: Risk management
An international standard that provides organisations with guidelines and principles for risk management within organisations

NCSC Risk Management Guidance
This guidance helps cyber security risk practitioners to better understand and manage the cyber security risks affecting their organisations.