Abstract:Deep neural networks can be easily fooled into making incorrect predictions through corruption of the input by adversarial perturbations: human-imperceptible artificial noise. So far adversarial training has been the most successful defense against such adversarial attacks. This work focuses on improving adversarial training to boost adversarial robustness. We first analyze, from an instance-wise perspective, how adversarial vulnerability evolves during adversarial training. We find that during training an overall reduction of adversarial loss is achieved by sacrificing a considerable proportion of training samples to be more vulnerable to adversarial attack, which results in an uneven distribution of adversarial vulnerability among data. Such "uneven vulnerability", is prevalent across several popular robust training methods and, more importantly, relates to overfitting in adversarial training. Motivated by this observation, we propose a new adversarial training method: Instance-adaptive Smoothness Enhanced Adversarial Training (ISEAT). It jointly smooths both input and weight loss landscapes in an adaptive, instance-specific, way to enhance robustness more for those samples with higher adversarial vulnerability. Extensive experiments demonstrate the superiority of our method over existing defense methods. Noticeably, our method, when combined with the latest data augmentation and semi-supervised learning techniques, achieves state-of-the-art robustness against $\ell_{\infty}$-norm constrained attacks on CIFAR10 of 59.32% for Wide ResNet34-10 without extra data, and 61.55% for Wide ResNet28-10 with extra data. Code is available at <a class="link-external link-https" href="https://github.com/TreeLLi/Instance-adaptive-Smoothness-Enhanced-AT" rel="external noopener nofollow">this https URL</a>.

Nrat: towards adversarial training with inherent label noise

NoiLIn: Improving Adversarial Training and Correcting Stereotype of Noisy Labels

Understanding the Interaction of Adversarial Training with Noisy Labels

Learning with Noisy Labels Via Self-supervised Adversarial Noisy Masking

Soften to Defend: Towards Adversarial Robustness via Self-Guided Label Refinement

DAT: Training Deep Networks Robust to Label-Noise by Matching the Feature Distributions

NAT: Noise-Aware Training for Robust Neural Sequence Labeling

Enhancing Robustness in Learning with Noisy Labels: an Asymmetric Co-Training Approach

Robust Learning with Adversarial Perturbations and Label Noise: A Two-Pronged Defense Approach.

Training Robust Deep Neural Networks via Adversarial Noise Propagation

Enhancing Robust Representation in Adversarial Training: Alignment and Exclusion Criteria

Dynamic Label Adversarial Training for Deep Learning Robustness Against Adversarial Attacks

Robust Learning with Adversarial Perturbations and Label Noise

ATAL: Active Learning Using Adversarial Training for Data Augmentation

Modeling Adversarial Noise for Adversarial Training.

Defense Against Adversarial Attacks Using Topology Aligning Adversarial Training

Improved Adversarial Training Through Adaptive Instance-wise Loss Smoothing

Adversarial Finetuning with Latent Representation Constraint to Mitigate Accuracy-Robustness Tradeoff

Effective and Robust Adversarial Training against Data and Label Corruptions

Robust Testing for Deep Learning using Human Label Noise

Progressively Robust Loss for Deep Learning with Noisy Labels