Machine learning principles

Like any other software, ML systems will contain vulnerabilities, and having proper vulnerability management processes will help address the risks associated with using them. The fast-developing nature of ML, and the increasing integration of ML components into critical systems, mean that sharing knowledge about vulnerabilities and incidents is particularly important.

Your ability to receive information from users about possible vulnerabilities in your own ML products will allow you to address security issues. In some sectors this may also help you comply with requirements or legislation linked to secure data handling.

Develop a vulnerability disclosure process for your system and organisation

Vulnerabilities are discovered all the time, and security specialists want to be able to report them directly to the organisation responsible. These reports can provide you with valuable information that you can use to improve the security of your systems.

By providing a clear process, organisations can receive the information directly so the vulnerability can be addressed, and the risk of compromise reduced. This process also reduces the reputational damage of public disclosure by providing a way to report, and a defined policy of how the organisation will respond. The NCSC's Vulnerability Disclosure Toolkit contains the main components required to set up your own vulnerability disclosure process.

Develop a process for responsibly sharing relevant threat intelligence

The NIST Guide to Threat Information Sharing provides guidance on establishing and participating in cyber threat information sharing relationships. Identify and share your knowledge at appropriate forums such as industry specific conferences, blogs and journals. 

Note that a real-life adversarial attack can be used to create a case study through MITRE ATLAS - Contribute. Principle 1.3 discusses benefits and risks of publishing specific detail about your systems. Nevertheless, there are likely to be various ways to share enough information to support others in developing secure ML, even if key sensitivities are removed.

Report incidents to relevant stakeholders and authorities

Follow the NCSC's advice for reporting and managing incidents and if appropriate report the incident to the NCSC. Where possible, share details about the incident to help build knowledge across the ML development community. One way to do this would be to create a case study in MITRE ATLAS, as discussed above.