Abstract:Deep neural networks can be easily fooled into making incorrect predictions through corruption of the input by adversarial perturbations: human-imperceptible artificial noise. So far adversarial training has been the most successful defense against such adversarial attacks. This work focuses on improving adversarial training to boost adversarial robustness. We first analyze, from an instance-wise perspective, how adversarial vulnerability evolves during adversarial training. We find that during training an overall reduction of adversarial loss is achieved by sacrificing a considerable proportion of training samples to be more vulnerable to adversarial attack, which results in an uneven distribution of adversarial vulnerability among data. Such "uneven vulnerability", is prevalent across several popular robust training methods and, more importantly, relates to overfitting in adversarial training. Motivated by this observation, we propose a new adversarial training method: Instance-adaptive Smoothness Enhanced Adversarial Training (ISEAT). It jointly smooths both input and weight loss landscapes in an adaptive, instance-specific, way to enhance robustness more for those samples with higher adversarial vulnerability. Extensive experiments demonstrate the superiority of our method over existing defense methods. Noticeably, our method, when combined with the latest data augmentation and semi-supervised learning techniques, achieves state-of-the-art robustness against $\ell_{\infty}$-norm constrained attacks on CIFAR10 of 59.32% for Wide ResNet34-10 without extra data, and 61.55% for Wide ResNet28-10 with extra data. Code is available at <a class="link-external link-https" href="https://github.com/TreeLLi/Instance-adaptive-Smoothness-Enhanced-AT" rel="external noopener nofollow">this https URL</a>.

Robustness through Cognitive Dissociation Mitigation in Contrastive Adversarial Training

Rethinking Robust Contrastive Learning from the Adversarial Perspective

Feature Augmentation for Adversarial Robustness

Robust Pre-Training by Adversarial Contrastive Learning

When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?

Towards Adversarial Robustness with Multidimensional Perturbations Via Contrastive Learning

Enhancing Robust Representation in Adversarial Training: Alignment and Exclusion Criteria

Rethinking the Effect of Data Augmentation in Adversarial Contrastive Learning

Towards Improving Robustness Against Common Corruptions in Object Detectors Using Adversarial Contrastive Learning

Improved Adversarial Training Through Adaptive Instance-wise Loss Smoothing

Adversarial Feature Alignment: Balancing Robustness and Accuracy in Deep Learning via Adversarial Training

The Importance of Robust Features in Mitigating Catastrophic Forgetting

Supervised Contrastive Prototype Learning: Augmentation Free Robust Neural Network

Adversarial Supervised Contrastive Learning

Splitting the Difference on Adversarial Training

Towards Adversarial Robust Representation Through Adversarial Contrastive Decoupling

CAT: Collaborative Adversarial Training.

CAT:Collaborative Adversarial Training

Feature Distillation With Guided Adversarial Contrastive Learning

Push Stricter to Decide Better: A Class-Conditional Feature Adaptive Framework for Improving Adversarial Robustness

Adversarial Contrastive Learning via Asymmetric InfoNCE.