Abstract:Despite remarkable achievements in deep learning across various domains, its inherent vulnerability to adversarial examples still remains a critical concern for practical deployment. Adversarial training has emerged as one of the most effective defensive techniques for improving model robustness against such malicious inputs. However, existing adversarial training schemes often lead to limited generalization ability against underlying adversaries with diversity due to their overreliance on a point-by-point augmentation strategy by mapping each clean example to its adversarial counterpart during training. In addition, adversarial examples can induce significant disruptions in the statistical information w.r.t. the target model, thereby introducing substantial uncertainty and challenges to modeling the distribution of adversarial examples. To circumvent these issues, in this paper, we propose a novel uncertainty-aware distributional adversarial training method, which enforces adversary modeling by leveraging both the statistical information of adversarial examples and its corresponding uncertainty estimation, with the goal of augmenting the diversity of adversaries. Considering the potentially negative impact induced by aligning adversaries to misclassified clean examples, we also refine the alignment reference based on the statistical proximity to clean examples during adversarial training, thereby reframing adversarial training within a distribution-to-distribution matching framework interacted between the clean and adversarial domains. Furthermore, we design an introspective gradient alignment approach via matching input gradients between these domains without introducing external models. Extensive experiments across four benchmark datasets and various network architectures demonstrate that our approach achieves state-of-the-art adversarial robustness and maintains natural performance.

Distributionally Robust Learning With Stable Adversarial Training

Stable Adversarial Learning under Distributional Shifts

Invariant Adversarial Learning for Distributional Robustness

Enhancing Adversarial Robustness via Uncertainty-Aware Distributional Adversarial Training

A Stochastic Subgradient Method for Distributionally Robust Non-Convex Learning

Asynchronous Byzantine-Robust Stochastic Aggregation with Variance Reduction for Distributed Learning

Distributionally Robust Machine Learning with Multi-source Data

Online Optimization and Ambiguity-Based Learning of Distributionally Uncertain Dynamic Systems

Stability Evaluation Through Distributional Perturbation Analysis

Robust Distributed Learning Against Both Distributional Shifts and Byzantine Attacks

Robust Distribution Learning with Local and Global Adversarial Corruptions

Stability Evaluation via Distributional Perturbation Analysis

Enhancing Distributional Stability among Sub-populations

Distributionally Robust Learning with Weakly Convex Losses: Convergence Rates and Finite-Sample Guarantees

A Robust Learning Approach for Regression Models Based on Distributionally Robust Optimization.

Fundamental Tradeoffs in Distributionally Adversarial Training

Statistical Learning of Distributionally Robust Stochastic Control in Continuous State Spaces

Incorporating Unlabeled Data into Distributionally Robust Learning

Does Distributionally Robust Supervised Learning Give Robust Classifiers?

Regularization for Adversarial Robust Learning