Abstract:The deep neural network (DNN) can be fooled by introducing a small perturbation to the example. Adversarial training (AT) is proven to be one of the most effective methods to improve robustness. Although the latest works to improve AT can ensure the adversarial robustness of PGD, the exploration of the performance under more disturbances is lacking, which may lead to our misunderstanding of model generalization. Moreover, they sacrifice DNN’s performance of classifying clean examples, which is called clean performance. Research shows that overfitting will be caused by the cross-mixing of distributions between natural and adversarial examples. We discover that the primary cause of cross-mixing is using the same batch-normalization (BN) layer to normalize the natural and adversarial examples because their means and variances are substantially different. We propose a new defense method which is called probabilistic distributions alignment adversarial training (PDAT). It utilizes the model with a multi-BN structure, which contains natural BN (NBN) and adversarial BN (ABN) to normalize the natural and adversarial examples separately and calculate their corresponding probabilistic distributions. It mutually trains the model through the alignment of probabilistic distributions between NBN and ABN. The natural training (NT) process of NBN is utilized to guide the training process of ABN which is much more difficult. To ensure that the model learns the correct knowledge during the alignment process, we use dynamic weights and soft-decision schemes in the loss function. This alleviates the cross-mixing problem. We conduct experiments on CIFAR10 and CIFAR100 and verify that PDAT improves robustness across a wide range of attacks while maximally maintaining clean performance. This proves that PDAT generalizes better.

Block-term Dropout for Robust Adversarial Defense

Attack As Defense: Characterizing Adversarial Examples Using Robustness.

Towards Robust DNNs: an Taylor Expansion-Based Method for Generating Powerful Adversarial Examples.

Defensive Dropout for Hardening Deep Neural Networks under Adversarial Attacks

Improving Model Robustness Against Adversarial Examples with Redundant Fully Connected Layer.

Deep Defense: Training DNNs with Improved Adversarial Robustness

An adversarial defense algorithm based on robust U-net

Active Dropblock: Method to Enhance Deep Model Accuracy and Robustness

TDNN:A Tensor Decomposition Adversarial Defense Method Based on Neural Network

DeepDefense: Training Deep Neural Networks with Improved Robustness.

AdvDrop: Adversarial Attack to DNNs by Dropping Information

Distributed Adversarial Training to Robustify Deep Neural Networks at Scale

Defending Adversarial Examples via DNN Bottleneck Reinforcement

Towards Deep Learning Models Resistant to Transfer-based Adversarial Attacks via Data-centric Robust Learning

Det: Defending Against Adversarial Examples Via Decreasing Transferability

Complete Defense Framework to Protect Deep Neural Networks Against Adversarial Examples

Robust Sparse Regularization: Defending Adversarial Attacks Via Regularized Sparse Network

Progressive Diversified Augmentation for General Robustness of DNNs: A Unified Approach.

Wasserstein distributional robustness of neural networks

Improve Adversarial Robustness Via Probabilistic Distributions Decoupled Network While Guaranteeing Clean Performance