Abstract:One major goal of the AI security community is to securely and reliably produce and deploy deep learning models for real-world applications. To this end, data poisoning based backdoor attacks on deep neural networks (DNNs) in the production stage (or training stage) and corresponding defenses are extensively explored in recent years. Ironically, backdoor attacks in the deployment stage, which can often happen in unprofessional users’ devices and are thus arguably far more threatening in real-world scenarios, draw much less attention of the community. We attribute this imbalance of vigilance to the weak practicality of existing deployment-stage backdoor attack algorithms and the insufficiency of real-world attack demonstrations. To fill the blank, in this work, we study the realistic threat of deployment-stage backdoor attacks on DNNs. We base our study on a commonly used deployment-stage attack paradigm — adversarial weight attack, where adversaries selectively modify model weights to embed backdoor into deployed DNNs. To approach realistic practicality, we propose the first gray-box and physically realizable weights attack algorithm for backdoor injection, namely subnet replacement attack (SRA), which only requires architecture information of the victim model and can support physical triggers in the real world. Extensive experimental simulations and system-level real-world attack demonstrations are conducted. Our results not only suggest the effectiveness and practicality of the proposed attack algorithm, but also reveal the practical risk of a novel type of computer virus that may widely spread and stealthily inject backdoor into DNN models in user devices. By our study, we call for more attention to the vulnerability of DNNs in the deployment stage.

DeepGuard: Backdoor Attack Detection and Identification Schemes in Privacy-Preserving Deep Neural Networks

Industrial Scale Privacy Preserving Deep Neural Network

Backdoor Attacks and Defenses for Deep Neural Networks in Outsourced Cloud Environments

MP-BADNet: A Backdoor-Attack Detection and Identification Protocol among Multi-Participants in Private Deep Neural Networks.

Defense-Resistant Backdoor Attacks Against Deep Neural Networks in Outsourced Cloud Environment

MP-BADNet + : Secure and effective backdoor attack detection and mitigation protocols among multi-participants in private DNNs

Adaptive Backdoor Attack Against Deep Neural Networks

Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks

Evading Backdoor Defenses: Concealing Genuine Backdoors Through Scapegoat Strategy

NTD: Non-Transferability Enabled Deep Learning Backdoor Detection

Regula Sub-rosa: Latent Backdoor Attacks on Deep Neural Networks

Mind Your Heart: Stealthy Backdoor Attack on Dynamic Deep Neural Network in Edge Computing

Backdoor Cleansing with Unlabeled Data

Confidence Matters: Inspecting Backdoors in Deep Neural Networks via Distribution Transfer

An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defences

Mind Your Heart: Stealthy Backdoor Attack on Dynamic Deep Neural Network in Edge Computing

Black-box Detection of Backdoor Attacks with Limited Information and Data

Backdoor Attacks to Deep Learning Models and Countermeasures: A Survey

Advancing Security in AI Systems: A Novel Approach to Detecting Backdoors in Deep Neural Networks

Efficient Backdoor Attacks for Deep Neural Networks in Real-world Scenarios

Backdoor Defense via Decoupling the Training Process