Abstract:Machine learning (ML) and artificial intelligence (AI) techniques have now become commonplace in software products and services. When threat modelling a system, it is therefore important that we consider threats unique to ML and AI techniques, in addition to threats to our software. In this paper, we present a threat model that can be used to systematically uncover threats to AI based software. The threat model consists of two main parts, a model of the software development process for AI based software and an attack taxonomy that has been developed using attacks found in adversarial AI research. We apply the threat model to two real life AI based software and discuss the process and the threats found.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the problem of the lack of a systematic approach in threat modeling for current artificial intelligence (AI) and machine learning (ML) software. Specifically, the author points out that although there are many studies on adversarial AI and ML, these studies do not provide a systematic method to identify threats in AI software. Therefore, when organizations assess the information security risks of the AI software they use, they often rely on inconsistent threat identification methods, which may be based on information provided by vendors or random threat enumeration. To solve this problem, the author proposes a systematic threat model that can help practitioners systematically identify and analyze potential threats in AI software. This threat model consists of two parts: 1. **Abstract model of the AI software development process**: It describes the main stages of AI software from data processing, model development to deployment. 2. **Attack taxonomy**: Based on existing adversarial AI research, it divides possible attacks into three major categories: - **Attacks on Dataset** - **Attacks on Model** - **Attacks on Input** By mapping these attack taxonomies to the AI software development process, the author hopes to provide a comprehensive and systematic framework to help organizations more effectively identify and respond to potential threats in AI software. ### Main contributions - **Systematic threat model**: It provides a systematic framework to help practitioners identify potential threats in AI software. - **Attack taxonomy**: It classifies in detail various types of attacks against AI software, including dataset attacks, model attacks, and input attacks. - **Practical application cases**: It verifies the effectiveness of the proposed threat model through two real - world AI software cases. ### Conclusion By proposing a systematic threat model, this paper fills the gap in existing literature in terms of AI software threat modeling and provides organizations with a more reliable and consistent threat identification method.

ADMIn: Attacks on Dataset, Model and Input. A Threat Model for AI Based Software

Towards more Practical Threat Models in Artificial Intelligence Security

Defense Against Adversarial Ai

Asset-centric Threat Modeling for AI-based Systems

Security and Privacy for Artificial Intelligence: Opportunities and Challenges

Taking off the Rose-Tinted Glasses: A Critical Look at Adversarial ML Through the Lens of Evasion Attacks

Security and Machine Learning in the Real World

Evaluating the Vulnerabilities in ML systems in terms of adversarial attacks

ML Attack Models: Adversarial Attacks and Data Poisoning Attacks

The Threat of Adversarial Attacks on Machine Learning in Network Security -- A Survey

Towards the Science of Security and Privacy in Machine Learning

Artificial Intelligence Security: Threats and Countermeasures

Countering Autonomous Cyber Threats

A Systematic Evaluation of Artificial Intelligence's Impact on the Landscape of Threat Modeling

Modeling Realistic Adversarial Attacks against Network Intrusion Detection Systems

Adversarial Attacks in Machine Learning: Key Insights and Defense Approaches

The Threat of Offensive AI to Organizations

Securing the Digital World: Protecting smart infrastructures and digital industries with Artificial Intelligence (AI)-enabled malware and intrusion detection

Modeling Threats to AI-ML Systems Using STRIDE

Exploring the Vulnerabilities of Machine Learning and Quantum Machine Learning to Adversarial Attacks using a Malware Dataset: A Comparative Analysis

Techniques for Adversarial Examples Threatening the Safety of Artificial Intelligence Based Systems