Abstract:With wide deployment of deep neural network (DNN) classifiers, there is great potential for harm from adversarial learning attacks. Recently, a special type of data poisoning (DP) attack, known as a backdoor (or Trojan), was proposed. These attacks do not seek to degrade classification accuracy, but rather to have the classifier learn to classify to a target class t∗ whenever the backdoor pattern is present in a test example originally from a source class s∗ . Launching backdoor attacks does not require knowledge of the classifier or its training process-only the ability to poison the training set with exemplars containing a backdoor pattern (labeled with the target class). Defenses against backdoors can be deployed before/during training, post-training, or at test time. Here, we address post-training detection in DNN image classifiers, seldom considered in existing works, wherein the defender does not have access to the poisoned training set, but only to the trained classifier itself, as well as to clean (unpoisoned) examples from the classification domain. This scenario is of great interest because e.g., a classifier may be the basis of a phone app that will be shared with many users. Detection may thus reveal a widespread attack. We propose a purely unsupervised anomaly detection (AD) defense against imperceptible backdoor attacks that: 1) detects whether the trained DNN has been backdoor-attacked; 2) infers the source and target classes in a detected attack; 3) estimates the backdoor pattern itself. Our AD approach involves learning (via suitable cost function minimization) the minimum size/norm perturbation (putative backdoor) required to induce the classifier to misclassify (most) examples from class s to class t , for all (s,t) pairs. Our hypothesis is that nonattacked pairs require large perturbations, while the attacked pair (s∗, t∗) requires much smaller ones. This is convincingly borne out experimentally. We identify a variety of plausible cost functions and devise a novel, robust hypothesis testing approach to perform detection inference. We test our approach, in comparison with the state-of-the-art methods, for several backdoor patterns, attack settings and mechanisms, and data sets and demonstrate its favorability. Our defense essentially requires setting a single hyperparameter (the detection threshold), which can e.g., be chosen to fix the system's false positive rate.

When Not to Classify: Detection of Reverse Engineering Attacks on DNN Image Classifiers

The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks

D-DAE: Defense-Penetrating Model Extraction Attacks.

ATTEQ-NN: Attention-based QoE-aware Evasive Backdoor Attacks.

When Not to Classify: Anomaly Detection of Attacks (ADA) on DNN Classifiers at Test Time

Universal Post-Training Reverse-Engineering Defense Against Backdoors in Deep Neural Networks

Reverse Engineering Imperceptible Backdoor Attacks on Deep Neural Networks for Detection and Training Set Cleansing

L-RED: Efficient Post-Training Detection of Imperceptible Backdoor Attacks without Access to the Training Set

Adversarial Learning Targeting Deep Neural Network Classification: A Comprehensive Review of Defenses Against Attacks

Breaking the False Sense of Security in Backdoor Defense through Re-Activation Attack

Invisible Backdoor Attack with Dynamic Triggers Against Person Re-identification

Rethinking the Reverse-engineering of Trojan Triggers

Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks

Look Closer to Your Enemy: Learning to Attack via Teacher-Student Mimicking

Need for Speed: Taming Backdoor Attacks with Speed and Precision

Reverse Engineering $\ell_p$ attacks: A block-sparse optimization approach with recovery guarantees

Regula Sub-rosa: Latent Backdoor Attacks on Deep Neural Networks

Detection of Backdoors in Trained Classifiers Without Access to the Training Set

Reverse Backdoor Distillation: Towards Online Backdoor Attack Detection for Deep Neural Network Models

Investigating the Backdoor on DNNs Based on Recolorization and Reconstruction: From a Multi-Channel Perspective

Evaluation of Defensive Methods for DNNs against Multiple Adversarial Evasion Models