Abstract:Deep learning models with backdoors act maliciously when triggered but seem normal otherwise. This risk, often increased by model outsourcing, challenges their secure use. Although countermeasures exist, their defense against adaptive attacks is under-examined, possibly leading to security misjudgments. This study is the first intricate examination illustrating the difficulty of detecting backdoors in outsourced models, especially when attackers adjust their strategies, even if their capabilities are significantly limited. It is relatively straightforward for attackers to circumvent detection by trivially violating its threat model (e.g., using advanced backdoor types or trigger designs not covered by the detection). However, this research highlights that various leading detection defenses can simultaneously be evaded using simple adaptive strategies, even under their defined threat models and with limited adversary capabilities (e.g., using easily detectable triggers while maintaining a high attack success rate). To be more specific, this study introduces a novel methodology that employs trigger specificity enhancement and training regulation in a symbiotic manner. This approach allows us to evade multiple backdoor detection defenses simultaneously, including Neural Cleanse (Oakland 19'), ABS (CCS 19'), and MNTD (Oakland 21'). These were the detection tools selected for the Evasive Trojans Track of the 2022 NeurIPS Trojan Detection Challenge. Even when applied in conjunction with these defenses under stringent conditions, such as a high attack success rate (> 97%) and the restricted use of the simplest trigger (small white square), our straightforward method garnered the second prize in NeurIPS Trojan Detection Challenge. Notably, for the first time, our adaptive attack successfully evaded other recent state-of-the-art defenses, including FeatureRE (NeurIPS 22') and Beatrix (NDSS 23'). This study suggests that existing model outsourcing backdoor defenses remain vulnerable to adaptive attacks, and thus, the use of third-party models should be avoided whenever possible.

SGBA: A Stealthy Scapegoat Backdoor Attack Against Deep Neural Networks

B3: Backdoor Attacks Against Black-box Machine Learning Models

Evading Backdoor Defenses: Concealing Genuine Backdoors Through Scapegoat Strategy

Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks

Stand-in Backdoor: A Stealthy and Powerful Backdoor Attack

Sparse Backdoor Attack Against Neural Networks.

Invisible Backdoor Attacks on Deep Neural Networks via Steganography and Regularization

Scalable Backdoor Detection in Neural Networks

Backdoor Attack with Sparse and Invisible Trigger

A Practical Trigger-Free Backdoor Attack on Neural Networks

PatchBackdoor: Backdoor Attack against Deep Neural Networks without Model Modification

Defense-Resistant Backdoor Attacks Against Deep Neural Networks in Outsourced Cloud Environment

On Model Outsourcing Adaptive Attacks to Deep Learning Backdoor Defenses

NTD: Non-Transferability Enabled Deep Learning Backdoor Detection

CatchBackdoor: Backdoor Detection via Critical Trojan Neural Path Fuzzing

BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain

Stealthy and Flexible Trojan in Deep Learning Framework

An Adaptive Black-box Backdoor Detection Method for Deep Neural Networks

A Novel Backdoor Attack Adapted to Transfer Learning.

Untargeted Backdoor Attack Against Object Detection

Regula Sub-rosa: Latent Backdoor Attacks on Deep Neural Networks