Machine learning principles

Guidelines for secure AI system development 
This document recommends guidelines for providers of any systems that use artificial intelligence (AI).

Secure-by-Design | CISA 
This joint guidance urges software manufacturers to take urgent steps necessary to ship products that are secure by design.

Machine Learning Security in Industry: A Quantitative Survey 
A report that sheds light on real-world attacks on deployed machine learning.

MITRE ATLAS: Craft Adversarial Data

MITRE ATLAS: LLM Prompt Injection

MITER ATLAS: Backdoor ML Model - Poison ML Model

MITER ATLAS: Exfiltration via ML Inference API: Infer Training Data Membership

MITER ATLAS: Exfiltration via ML Inference API: Invert ML Model 

Failure Modes in Machine Learning 
A blog from Microsoft. 

NIST Artificial Intelligence (AI) 100-2 E2023, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations 
This NIST Trustworthy and Responsible AI report develops a taxonomy of concepts and defines terminology in the field of adversarial machine learning (AML).

AI Security Concerns in a Nutshell (PDF)
This guideline from Federal Office for Information Security introduces developers to the most relevant attacks on machine learning systems and potential complementary defences.

Developing a positive cyber security culture 
A chapter from the NCSC’s Board Toolkit, which helps board members to govern cyber risk more effectively.

Secure development and deployment guidance 
The NCSC's 8 Principles to help you improve and evaluate your development practices, and those of your suppliers.

Multilayer Framework for Good Cybersecurity Practices for AI 
ENISA’s  scalable framework to guide NCAs and AI stakeholders on the steps they need to follow to secure their AI systems.

Threat Modeling | OWASP Foundation
The OWASP® Foundation works to improve the security of software through its community-led open source software projects.

Election Security Spotlight – CIA Triad 
The CIA Triad is a benchmark model in information security designed to govern and evaluate how an organization handles data when it is stored, transmitted, or processed.

Data Science and Engineering With Human in the Loop, Behind the Loop, and Above the Loop  Issue 5.2, Spring 2023 
Editorial from Xiao-Li Meng, Whipple V. N. Jones Professor of Statistics, Harvard University.

Risk management 
NCSC’s guidance to help you better understand and manage the cyber security risks affecting your organisation.

NIST’s Artificial Intelligence Risk Management Framework (PDF)
Designed to equip organizations and individuals with approaches that increase the trustworthiness of AI systems.

Exercise caution when building off LLMs 
NCSC blog outlining the risks of building services that use LLMs.

MITRE ATLAS: ProofPoint Evasion

MITRE ATLAS: GPT-2 Model Replication

Google's AI Red Team: the ethical hackers making AI safer 
Blog introducing Google’s AI Red Team.

Ministry of Defence red teaming handbook
A practical guide for supporting individuals and teams who are faced with different problems and challenges in defence.

LVE Repository 
Documents and track vulnerabilities and exposures of large language models (LVEs).

The DARPA GARD
A selection of tools that seek to establish theoretical ML system foundations to identify system vulnerabilities.

Azure Counterfit
A generic automation layer for assessing the security of machine learning systems. 

CleverHans
A Python library to benchmark machine learning systems' vulnerability to adversarial examples.

AI Verify
An AI governance testing framework and software toolkit.

AI Standards Hub 
An initiative dedicated to the evolving field of standardisation for AI technologies.

MITRE ATLAS: Poison Data Training

MITRE ATLAS: PoisonGPT

Never a dill moment: Exploiting machine learning pickle files 
A blog discussing the underhanded antics that can occur simply from loading an untrusted pickle file or ML model.

ImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks (PDF).
Paper exploring how backdoors can be added during compilation, circumventing any safeguards in the data-preparation and model-training stages.

Supply chain security guidance
The NCSC's essential information, guidance and advice on securing your supply chain.

Safeguarding artifact integrity across any software supply chain
A specification for describing and incrementally improving supply chain security, established by industry consensus.

CycloneDX - Machine Learning Bill of Materials (ML-BOM) 
Model and dataset transparency for security, privacy, safety and ethical considerations.

HuggingFace platform: security
The Hugging Face Hub offers several security features to ensure that your code and data are secure.

https://www.turing.ac.uk/blog/what-synthetic-data-and-how-can-it-advance-research-and-development 
Research carried out by The Alan Turing Institute.

Machine learning with limited data 
A handbook from the Defence Science and Technology Library (DSTL).

Creating instructions for human labellers  |  Data Labelling Service  |  Google Cloud 
Data labelling guidance from Google.

Data-at-rest protection 
NCSC guidance for protecting data-at-rest.

Data-in-transit protection 
NCSC guidance for protecting data-in-transit.

ISO/IEC 27001:2022 
The globally used standard for information security management systems.

Device Security Guidance
NCSC guidance for organisations on how to choose, configure and use devices securely. 

CIS Benchmarks™ 
Configuration recommendations to help you protect your systems against threats.

GitHub - cisagov/LME: Logging Made Easy (LME) is a no-cost and open logging and protective monitoring solution serving all organizations. 
Logging software maintained by CISA.

Introduction to logging for security purposes 
The NCSC guidance to help you devise an approach to logging.

What are you protecting? 
The NCSC's 15 good practice measures for the protection of bulk data held by digital services.

UK GDPR guidance and resources 
GDPR guidance and resources from the Information Commissioner’s Office.

Git 
A free and open source distributed version control system.

How GDS uses git and GitHub 
A blog explaining how the UK government uses GitHub.

Responsible AI: The Role of Data and Model Cards 
How to use data cards and model cards to bring greater transparency between stakeholders and model development teams.

“Guide of Introduction of Software Bill of Materials (SBOM) for Software Management” Formulated 
A guide mainly targeting software suppliers as a compilation of the advantages of introducing SBOM to companies.

Enabling AI with Data Cards 
Guidance from US Chief Digital And AI Office.

GitHub - gchq/Bailo: Managing the lifecycle of machine learning to support scalability, impact, collaboration, compliance and sharing. 
Software maintained by GCHQ.

Machine Learning: The High-Interest Credit Card of Technical Debt (PDF)
A paper explaining several machine learning specific risk factors and design patterns to be avoided or refactored where possible.

Models - Hugging Face 
A platform where the machine learning community collaborates on models, datasets, and applications.

Pickle Scanning 
A widely used serialization format in ML.

Tutorial: Learning Curves for Machine Learning in Python for Data Science 
Reducing bias and variance to build more accurate models.

Pruning Tutorial — PyTorch Tutorials 2.3.0+cu121 documentation 
How to use torch.nn.utils.prune to sparsify your neural networks.
 

MITRE ATLAS: Defense Evasion

MITRE ATLAS: Exfiltration via ML Inference API: Extract ML Model

MITRE ATLAS: LLM Meta Prompt Extraction

Exploring Prompt Injection Attacks 
A blog from NCC group discussing what a prompt is in the machine learning context.

MITRE ATLAS: TAY POISONING

ml-ops.org 
MLOps strive to avoid ‘technical debt’ in machine learning applications.

CML · Continuous Machine Learning 
Continuous Machine Learning (CML) is CI/CD for Machine Learning Projects.

MLflow | MLflow 
Build better models and generative AI apps on a unified, end-to-end,
open source MLOps platform.

Privacy-Preserving Federated Learning: Understanding the Costs and Benefits 
Examples of the costs and benefits associated with applying PETs to enable privacy-preserving federated learning.

What is Out-of-Distribution (OOD) Detection? 
Approaches to Detect OOD Instances from Encord.

Vulnerability Disclosure Toolkit 
The NCSC's toolkit that makes it easier for organisations to create a vulnerability disclosure process.

NIST’s Guide to Cyber Threat Information Sharing (PDF)
Provides guidelines for establishing and participating in cyber threat information sharing relationships.

Incident management 
Resources from the NCSC helping to reduce the harm from cyber security incidents in the UK.

Secure sanitisation of storage media 
The NCSC's guidance on why sanitisation is necessary, the risks involved, and how to sanitise affordably.

Guide to archiving personal data (PDF)
Guidance from the National Archives.