Abstract:The deep neural network (DNN) can be fooled by introducing a small perturbation to the example. Adversarial training (AT) is proven to be one of the most effective methods to improve robustness. Although the latest works to improve AT can ensure the adversarial robustness of PGD, the exploration of the performance under more disturbances is lacking, which may lead to our misunderstanding of model generalization. Moreover, they sacrifice DNN’s performance of classifying clean examples, which is called clean performance. Research shows that overfitting will be caused by the cross-mixing of distributions between natural and adversarial examples. We discover that the primary cause of cross-mixing is using the same batch-normalization (BN) layer to normalize the natural and adversarial examples because their means and variances are substantially different. We propose a new defense method which is called probabilistic distributions alignment adversarial training (PDAT). It utilizes the model with a multi-BN structure, which contains natural BN (NBN) and adversarial BN (ABN) to normalize the natural and adversarial examples separately and calculate their corresponding probabilistic distributions. It mutually trains the model through the alignment of probabilistic distributions between NBN and ABN. The natural training (NT) process of NBN is utilized to guide the training process of ABN which is much more difficult. To ensure that the model learns the correct knowledge during the alignment process, we use dynamic weights and soft-decision schemes in the loss function. This alleviates the cross-mixing problem. We conduct experiments on CIFAR10 and CIFAR100 and verify that PDAT improves robustness across a wide range of attacks while maximally maintaining clean performance. This proves that PDAT generalizes better.

Weighted Adaptive Perturbations Adversarial Training for Improving Robustness

GAAT: Group Adaptive Adversarial Training to Improve the Trade-Off Between Robustness and Accuracy

Adversarial Weight Perturbation Helps Robust Generalization

Strength-Adaptive Adversarial Training

Improving Accuracy-robustness Trade-off via Pixel Reweighted Adversarial Training

Feature Augmentation for Adversarial Robustness

Improving Adversarial Training using Vulnerability-Aware Perturbation Budget

Enhancing Adversarial Training via Reweighting Optimization Trajectory

Improved Adversarial Training Through Adaptive Instance-wise Loss Smoothing

Robust Training with Feature-Based Adversarial Example

ATRA: Efficient Adversarial Training with High-Robust Area

Towards Adversarial Robustness with Multidimensional Perturbations Via Contrastive Learning

A2: Efficient Automated Attacker for Boosting Adversarial Training

Robust Weight Perturbation for Adversarial Training

Improving Adversarial Robustness via Attention and Adversarial Logit Pairing

Enhancing Adversarial Robustness Via Anomaly-aware Adversarial Training.

Two-Way Feature-Aligned and Attention-Rectified Adversarial Training

CAT: Customized Adversarial Training for Improved Robustness

Improve Adversarial Robustness Via Probabilistic Distributions Decoupled Network While Guaranteeing Clean Performance

Adversarial Training of Deep Neural Networks Guided by Texture and Structural Information

Adversarial Training on Purification (AToP): Advancing Both Robustness and Generalization