Abstract:It is becoming increasingly clear that many machine learning classifiers are vulnerable to adversarial examples. In attempting to explain the origin of adversarial examples, previous studies have typically focused on the fact that neural networks operate on high dimensional data, they overfit, or they are too linear. Here we argue that the origin of adversarial examples is primarily due to an inherent uncertainty that neural networks have about their predictions. We show that the functional form of this uncertainty is independent of architecture, dataset, and training protocol; and depends only on the statistics of the logit differences of the network, which do not change significantly during training. This leads to adversarial error having a universal scaling, as a power-law, with respect to the size of the adversarial perturbation. We show that this universality holds for a broad range of datasets (MNIST, CIFAR10, ImageNet, and random data), models (including state-of-the-art deep networks, linear models, adversarially trained networks, and networks trained on randomly shuffled labels), and attacks (FGSM, step l.l., PGD). Motivated by these results, we study the effects of reducing prediction entropy on adversarial robustness. Finally, we study the effect of network architectures on adversarial sensitivity. To do this, we use neural architecture search with reinforcement learning to find adversarially robust architectures on CIFAR10. Our resulting architecture is more robust to white \emph{and} black box attacks compared to previous attempts.

Understanding adversarial examples requires a theory of artefacts for deep learning

Attack As Defense: Characterizing Adversarial Examples Using Robustness.

Intriguing Properties of Adversarial Examples

Adversarial Examples: Opportunities and Challenges

Adversarial Examples on Object Recognition: A Comprehensive Survey

Adversarial examples: attacks and defenses in the physical world

Adversarial Examples: Attacks and Defenses for Deep Learning

Adversarial Examples in Physical World

Interpreting Adversarial Examples in Deep Learning: A Review

Are adversarial examples inevitable?

Adversarial Attacks Hidden in Plain Sight

Adversarial Examples in Deep Learning: Characterization and Divergence

How Can We Deal with Adversarial Examples?

Adversarial examples: A survey of attacks and defenses in deep learning-enabled cybersecurity systems

Adversarial Examples Are Not Real Features

Adversarial Example Games

Developing and Defeating Adversarial Examples

Adversarial Attack? Don't Panic

Research Progress and Challenges on Application-Driven Adversarial Examples: A Survey

Motivating the rules of the game for adversarial example research