Abstract:With wide deployment of machine learning (ML)-based systems for a variety of applications including medical, military, automotive, genomic, multimedia, and social networking, there is great potential for damage from adversarial learning (AL) attacks. In this article, we provide a contemporary survey of AL, focused particularly on defenses against attacks on deep neural network classifiers. After introducing relevant terminology and the goals and range of possible knowledge of both attackers and defenders, we survey recent work on test-time evasion (TTE), data poisoning (DP), backdoor DP, and reverse engineering (RE) attacks and particularly defenses against the same. In so doing, we distinguish robust classification from anomaly detection (AD), unsupervised from supervised, and statistical hypothesis-based defenses from ones that do not have an explicit null (no attack) hypothesis. We also consider several scenarios for detecting backdoors. We provide a technical assessment for reviewed works, including identifying any issues/limitations, required hyperparameters, needed computational complexity, as well as the performance measures evaluated and the obtained quality. We then delve deeper, providing novel insights that challenge conventional AL wisdom and that target unresolved issues, including: robust classification versus AD as a defense strategy; the belief that attack success increases with attack strength, which ignores susceptibility to AD; small perturbations for TTE attacks: a fallacy or a requirement; validity of the universal assumption that a TTE attacker knows the ground-truth class for the example to be attacked; black, gray, or white-box attacks as the standard for defense evaluation; and susceptibility of query-based RE to an AD defense. We also discuss attacks on the privacy of training data. We then present benchmark comparisons of several defenses against TTE, RE, and backdoor DP attacks -n images. The article concludes with a discussion of continuing research directions, including the supreme challenge of detecting attacks whose goal is not to alter classification decisions, but rather simply to embed, without detection, "fake news" or other false content.

Trust but Verify: An Information-Theoretic Explanation for the Adversarial Fragility of Machine Learning Systems, and a General Defense against Adversarial Attacks

Attack As Defense: Characterizing Adversarial Examples Using Robustness.

Towards unlocking the mystery of adversarial fragility of neural networks

Towards Robustifying Image Classifiers against the Perils of Adversarial Attacks on Artificial Intelligence Systems

How adversarial attacks can disrupt seemingly stable accurate classifiers

Security Matters: A Survey on Adversarial Machine Learning

Intriguing Properties of Adversarial Examples

Defenses in Adversarial Machine Learning: A Survey

A practical approach to evaluating the adversarial distance for machine learning classifiers

A Survey of Robustness and Safety of 2D and 3D Deep Learning Models Against Adversarial Attacks

Defense Against Adversarial Ai

A Review of Adversarial Attack and Defense for Classification Methods

Simple Transparent Adversarial Examples

Adversarial Attacks and Defenses in Machine Learning-Powered Networks: A Contemporary Survey

Adversarial Learning in Statistical Classification: A Comprehensive Review of Defenses Against Attacks

Law and Adversarial Machine Learning

Adversarial Robustness of Deep Neural Networks: A Survey from a Formal Verification Perspective

Evaluating and Improving Adversarial Robustness of Machine Learning-Based Network Intrusion Detectors

Towards Robustness against Unsuspicious Adversarial Examples

A Survey on Adversarial Machine Learning for Code Data: Realistic Threats, Countermeasures, and Interpretations

Adversarial Learning Targeting Deep Neural Network Classification: A Comprehensive Review of Defenses Against Attacks