Abstract:Due to numerous breakthroughs in real-world applications brought by machine intelligence, deep neural networks (DNNs) are widely employed in critical applications. However, predictions of DNNs are easily manipulated with imperceptible adversarial perturbations, which impedes the further deployment of DNNs and may result in profound security and privacy implications. By incorporating adversarial samples into the training data pool, adversarial training is the strongest principled strategy against various adversarial attacks among all sorts of defense methods. Recent works mainly focus on developing new loss functions or regularizers, attempting to find the unique optimal point in the weight space. But none of them taps the potentials of classifiers obtained from standard adversarial training, especially states on the searching trajectory of training. In this work, we are dedicated to the weight states of models through the training process and devise a simple but powerful \emph{Self-Ensemble Adversarial Training} (SEAT) method for yielding a robust classifier by averaging weights of history models. This considerably improves the robustness of the target model against several well known adversarial attacks, even merely utilizing the naive cross-entropy loss to supervise. We also discuss the relationship between the ensemble of predictions from different adversarially trained models and the prediction of weight-ensembled models, as well as provide theoretical and empirical evidence that the proposed self-ensemble method provides a smoother loss landscape and better robustness than both individual models and the ensemble of predictions from different classifiers. We further analyze a subtle but fatal issue in the general settings for the self-ensemble model, which causes the deterioration of the weight-ensembled method in the late phases.

Adversarial training via multi-guidance and historical memory enhancement

GAAT: Group Adaptive Adversarial Training to Improve the Trade-Off Between Robustness and Accuracy

Learn from the Past: A Proxy based Adversarial Defense Framework to Boost Robustness

Enhancing adversarial robustness for deep metric learning via neural discrete adversarial training

Feature Augmentation for Adversarial Robustness

An adversarial defense algorithm based on robust U-net

Mutual Adversarial Training: Learning together is better than going alone

Enhancing Robust Representation in Adversarial Training: Alignment and Exclusion Criteria

Boosting Adversarial Robustness Via Self-Paced Adversarial Training.

Boosting Adversarial Training with Hardness-Guided Attack Strategy

Improve Adversarial Robustness Via Probabilistic Distributions Decoupled Network While Guaranteeing Clean Performance

Dual Head Adversarial Training.

Efficient Two-Step Adversarial Defense for Deep Neural Networks

Self-ensemble Adversarial Training for Improved Robustness

Learning More Robust Features with Adversarial Training

Improving Adversarial Robustness via Attention and Adversarial Logit Pairing

Efficient Adversarial Training with Multi-Fidelity Optimization for Robust Neural Network

A3GT: An Adaptive Asynchronous Generalized Adversarial Training Method

Improving adversarial robustness of deep neural networks by using semantic information

DeepDefense: Training Deep Neural Networks with Improved Robustness.

Guidance Through Surrogate: Toward a Generic Diagnostic Attack