Abstract:Adversarial training has achieved remarkable advancements in defending against adversarial attacks. Among them, fast adversarial training (FAT) is gaining attention for its ability to achieve competitive robustness with fewer computing resources. Existing FAT methods typically employ a uniform strategy that optimizes all training data equally without considering the influence of different examples, which leads to an imbalanced optimization. However, this imbalance remains unexplored in the field of FAT. In this paper, we conduct a comprehensive study of the imbalance issue in FAT and observe an obvious class disparity regarding their performances. This disparity could be embodied from a perspective of alignment between clean and robust accuracy. Based on the analysis, we mainly attribute the observed misalignment and disparity to the imbalanced optimization in FAT, which motivates us to optimize different training data adaptively to enhance robustness. Specifically, we take disparity and misalignment into consideration. First, we introduce self-knowledge guided regularization, which assigns differentiated regularization weights to each class based on its training state, alleviating class disparity. Additionally, we propose self-knowledge guided label relaxation, which adjusts label relaxation according to the training accuracy, alleviating the misalignment and improving robustness. By combining these methods, we formulate the Self-Knowledge Guided FAT (SKG-FAT), leveraging naturally generated knowledge during training to enhance the adversarial robustness without compromising training efficiency. Extensive experiments on four standard datasets demonstrate that the SKG-FAT improves the robustness and preserves competitive clean accuracy, outperforming the state-of-the-art methods.

Improving Fast Adversarial Training via Self-Knowledge Guidance

Improving Fast Adversarial Training with Prior-Guided Knowledge

Bag of Tricks for FGSM Adversarial Training

Revisiting and Exploring Efficient Fast Adversarial Training via LAW: Lipschitz Regularization and Auto Weight Averaging

GAAT: Group Adaptive Adversarial Training to Improve the Trade-Off Between Robustness and Accuracy

GEAR: A Margin-based Federated Adversarial Training Approach

Fast Adversarial Training with Smooth Convergence

Improving Fast Adversarial Training Paradigm: An Example Taxonomy Perspective

Preventing Catastrophic Overfitting in Fast Adversarial Training: A Bi-level Optimization Perspective

Prior-Guided Adversarial Initialization for Fast Adversarial Training

Boosting Adversarial Training in Safety-Critical Systems Through Boundary Data Selection

Push Stricter to Decide Better: A Class-Conditional Feature Adaptive Framework for Improving Adversarial Robustness

Improving Robust Fairness via Balance Adversarial Training

Boosting Adversarial Robustness Via Self-Paced Adversarial Training.

Attacks Which Do Not Kill Training Make Adversarial Learning Stronger

Towards Fairness-Aware Adversarial Learning

Fast Adversarial Training against Textual Adversarial Attacks

Initializing Perturbations in Multiple Directions for Fast Adversarial Training

CalFAT: Calibrated Federated Adversarial Training with Label Skewness

Robust Overfitting Does Matter: Test-Time Adversarial Purification With FGSM

Catastrophic Overfitting: A Potential Blessing in Disguise