Abstract:In order to deploy deep neural networks (DNNs) in high-stakes scenarios, it is imperative that DNNs provide inference robust to external perturbations - both intentional and unintentional. Although the resilience of DNNs to intentional and unintentional perturbations has been widely investigated, a unified vision of these inherently intertwined problem domains is still missing. In this work, we fill this gap by providing a survey of the state of the art and highlighting the similarities of the proposed approaches.We also analyze the research challenges that need to be addressed to deploy resilient and secure DNNs. As there has not been any such survey connecting the resilience of DNNs to intentional and unintentional perturbations, we believe this work can help advance the frontier in both domains by enabling the exchange of ideas between the two communities.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the robustness and security problems of deep neural networks (DNNs) in the face of intentional and unintentional perturbations. Specifically, the goals of the paper include: 1. **Provide a unified perspective**: Although there has been a lot of research on intentional and unintentional perturbations, there is currently a lack of a comprehensive perspective that combines these two areas. This paper fills this gap by reviewing existing research and pointing out the similarities in these areas. 2. **Analyze current research challenges**: The paper deeply analyzes the key challenges that need to be solved in current research in order to deploy robust and secure DNNs. This includes how to make DNNs robust to various external perturbations in high - risk scenarios. 3. **Promote communication between the two communities**: By connecting the research on intentional perturbations (such as adversarial sample attacks) and unintentional perturbations (such as out - of - distribution sample detection), the author hopes to promote the exchange of ideas between these two research communities, thereby promoting the development of their respective fields. ### Specific background and problem description - **Intentional perturbations (adversarial attacks)**: These perturbations are deliberately introduced by attackers to mislead the output of DNNs. For example, by making small modifications to the input image (such as changing a few pixels), the DNN can be misclassified. Common adversarial attack methods include: - **Non - target attack**: Does not specify a specific target category, only requires misclassification. \[ f(x + \delta) \neq y, \quad \|\delta\| \leq \epsilon \] - **Target attack**: Induces the DNN to misclassify the input as a specific target category \(\hat{y} \neq y\). \[ f(x + \delta) = \hat{y}, \quad \|\delta\| \leq \epsilon \] - **Unintentional perturbations (out - of - distribution samples)**: These perturbations are caused by natural phenomena or environmental changes, such as weather conditions (frost, rain, etc.), sensor noise, blurring, etc. In addition, it also includes changes in the label distribution (such as semantic drift). Specific types include: - **Covariate shift**: The input distribution changes, but the label distribution remains unchanged. - **Semantic shift**: Both the input and label distributions change, introducing new categories. - **Prior shift**: The label distribution changes, but the data distribution for a given label remains unchanged. - **Generalized label shift**: Both covariate shift and prior shift occur simultaneously. ### Main contributions of the paper - **Classification and discussion**: The paper classifies and discusses the classic, important, and recent work in the fields of OOD detection (unintentional interference) and adversarial sample detection (intentional interference). - **Commonalities and defense strategies**: Studies the commonalities of intentional and unintentional perturbation detection and their corresponding defense strategies, and points out the advantages and disadvantages of these methods. - **Open problems and research directions**: Proposes open problems and future research directions to ensure the robustness of DNNs in practical inference systems. Through these efforts, the paper provides a comprehensive review of the research on intentional and unintentional perturbations and points out the directions for further research.

Resilience and Security of Deep Neural Networks Against Intentional and Unintentional Perturbations: Survey and Research Challenges

When Single Event Upset Meets Deep Neural Networks: Observations, Explorations, and Remedies

Industrial Scale Privacy Preserving Deep Neural Network

There is Limited Correlation Between Coverage and Robustness for Deep Neural Networks

An Empirical Study on Correlation between Coverage and Robustness for Deep Neural Networks

A Survey of Neural Network Robustness Assessment in Image Recognition

Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety

A Survey of Trojan Attacks and Defenses to Deep Neural Networks

An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defences

Towards Security Threats of Deep Learning Systems: A Survey

Adversarial robustness improvement for deep neural networks

Physical Side-Channel Attacks on Embedded Neural Networks: A Survey

SoK: Certified Robustness for Deep Neural Networks

Adversarial Attacks and Defenses Against Deep Neural Networks: A Survey

Backdoor Attacks to Deep Neural Networks: A Survey of the Literature, Challenges, and Future Research Directions

Cybersecurity in Neural Interfaces: Survey and Future Trends

Resilience of Entropy Model in Distributed Neural Networks

Privacy Inference Attacks and Defenses in Cloud-based Deep Neural Network: A Survey

Towards Trustworthy Deep Learning

A survey on deep learning for cybersecurity: Progress, challenges, and opportunities