Abstract:State-of-the-art neural network models are actively used in various fields, but it is well-known that they are vulnerable to adversarial example attacks. Throughout the efforts to make the models robust against adversarial example attacks, it has been found to be a very difficult task. While many defense approaches were shown to be not effective, adversarial training remains as one of the promising methods. In adversarial training, the training data are augmented by “adversarial” samples generated using an attack algorithm. If the attacker uses a similar attack algorithm to generate adversarial examples, the adversarially trained network can be quite robust to the attack. However, there are numerous ways of creating adversarial examples, and the defender does not know what algorithm the attacker may use. A natural question is: Can we use adversarial training to train a model robust to multiple types of attack? Previous work have shown that, when a network is trained with adversarial examples generated from multiple attack methods, the network is still vulnerable to white-box attacks where the attacker has complete access to the model parameters. In this paper, we study this question in the context of black-box attacks, which can be a more realistic assumption for practical applications. Experiments with the MNIST dataset show that adversarially training a network with an attack method helps defending against that particular attack method, but has limited effect for other attack methods. In addition, even if the defender trains a network with multiple types of adversarial examples and the attacker attacks with one of the methods, the network could lose accuracy to the attack if the attacker uses a different data augmentation strategy on the target network. These results show that it is very difficult to make a robust network using adversarial training, even for black-box settings where the attacker has restricted information on the target network.

Minimizing Adversarial Training Samples for Robust Image Classifiers: Analysis and Adversarial Example Generator Design

Attack As Defense: Characterizing Adversarial Examples Using Robustness.

An adversarial defense algorithm based on robust U-net

Robust Attack with Adaptive Compress Adversarial Perturbations

Improving Adversarial Robustness Requires Revisiting Misclassified Examples.

An Efficient Framework for Generating Robust Adversarial Examples.

Robust Adversarial Examples Against Scale Transformation Via Generative Network

Detecting Adversarial Image Examples in Deep Neural Networks with Adaptive Noise Reduction

Secure Machine Learning Against Adversarial Samples at Test Time

On the Effectiveness of Adversarial Training in Defending against Adversarial Example Attacks for Image Classification

Are You Confident That You Have Successfully Generated Adversarial Examples?

Improving adversarial robustness of deep neural networks by using semantic information

ATRA: Efficient Adversarial Training with High-Robust Area

DeepDefense: Training Deep Neural Networks with Improved Robustness.

Robust Machine Learning Against Adversarial Samples at Test Time

An efficient adversarial example generation algorithm based on an accelerated gradient iterative fast gradient

Adversarial Defense Via the Data-Dependent Activation, Total Variation Minimization, and Adversarial Training

Adversarial Image Generation and Training for Deep Convolutional Neural Networks.

Feature Denoising for Improving Adversarial Robustness

Generating Adversarial Examples with Adversarial Networks

AdvJND: Generating Adversarial Examples with Just Noticeable Difference