Abstract:The deep neural network (DNN) can be fooled by introducing a small perturbation to the example. Adversarial training (AT) is proven to be one of the most effective methods to improve robustness. Although the latest works to improve AT can ensure the adversarial robustness of PGD, the exploration of the performance under more disturbances is lacking, which may lead to our misunderstanding of model generalization. Moreover, they sacrifice DNN’s performance of classifying clean examples, which is called clean performance. Research shows that overfitting will be caused by the cross-mixing of distributions between natural and adversarial examples. We discover that the primary cause of cross-mixing is using the same batch-normalization (BN) layer to normalize the natural and adversarial examples because their means and variances are substantially different. We propose a new defense method which is called probabilistic distributions alignment adversarial training (PDAT). It utilizes the model with a multi-BN structure, which contains natural BN (NBN) and adversarial BN (ABN) to normalize the natural and adversarial examples separately and calculate their corresponding probabilistic distributions. It mutually trains the model through the alignment of probabilistic distributions between NBN and ABN. The natural training (NT) process of NBN is utilized to guide the training process of ABN which is much more difficult. To ensure that the model learns the correct knowledge during the alignment process, we use dynamic weights and soft-decision schemes in the loss function. This alleviates the cross-mixing problem. We conduct experiments on CIFAR10 and CIFAR100 and verify that PDAT improves robustness across a wide range of attacks while maximally maintaining clean performance. This proves that PDAT generalizes better.

GAAT: Group Adaptive Adversarial Training to Improve the Trade-Off Between Robustness and Accuracy

Towards Desirable Decision Boundary by Moderate-Margin Adversarial Training

Blind Adversarial Training: Balance Accuracy and Robustness

Feature Augmentation for Adversarial Robustness

Strength-Adaptive Adversarial Training

CAT: Collaborative Adversarial Training.

CAT:Collaborative Adversarial Training

Blind Adversarial Training: Towards Comprehensively Robust Models Against Blind Adversarial Attacks.

ATRA: Efficient Adversarial Training with High-Robust Area

GAT: Guided Adversarial Training with Pareto-optimal Auxiliary Tasks

Improving Accuracy-robustness Trade-off via Pixel Reweighted Adversarial Training

Improve Adversarial Robustness Via Probabilistic Distributions Decoupled Network While Guaranteeing Clean Performance

Improving Adversarial Robustness via Attention and Adversarial Logit Pairing

Towards sustainable adversarial training with successive perturbation generation

CAT: Customized Adversarial Training for Improved Robustness

On the Tradeoff Between Robustness and Fairness

Weighted Adaptive Perturbations Adversarial Training for Improving Robustness

Adversarial Robustness under Long-Tailed Distribution Supplementary Material

Adversarial Robustness Overestimation and Instability in TRADES

Boosting Adversarial Robustness Via Self-Paced Adversarial Training.

Adversarial training via multi-guidance and historical memory enhancement