Abstract:Recently proposed adversarial training methods show the robustness to both adversarial and original examples and achieve state-of-the-art results in supervised and semi-supervised learning. All the existing adversarial training methods consider only how the worst perturbed examples (i.e., adversarial examples) could affect the model output. Despite their success, we argue that such setting may be in lack of generalization, since the output space (or label space) is apparently less informative.In this paper, we propose a novel method, called Manifold Adversarial Training (MAT). MAT manages to build an adversarial framework based on how the worst perturbation could affect the distributional manifold rather than the output space. Particularly, a latent data space with the Gaussian Mixture Model (GMM) will be first derived.On one hand, MAT tries to perturb the input samples in the way that would rough the distributional manifold the worst. On the other hand, the deep learning model is trained trying to promote in the latent space the manifold smoothness, measured by the variation of Gaussian mixtures (given the local perturbation around the data point). Importantly, since the latent space is more informative than the output space, the proposed MAT can learn better a robust and compact data representation, leading to further performance improvement. The proposed MAT is important in that it can be considered as a superset of one recently-proposed discriminative feature learning approach called center loss. We conducted a series of experiments in both supervised and semi-supervised learning on three benchmark data sets, showing that the proposed MAT can achieve remarkable performance, much better than those of the state-of-the-art adversarial approaches. We also present a series of visualization which could generate further understanding or explanation on adversarial examples.

Generalization Analysis of Adversarial Pairwise Learning

Generalization in Generative Adversarial Networks: A Novel Perspective from Privacy Protection.

Online Learning with Pairwise Loss Functions

Theoretical Analysis of Adversarial Learning: A Minimax Approach

Generalization Bounds for Adversarial Contrastive Learning

Stability Analysis and Generalization Bounds of Adversarial Training

Manifold Adversarial Learning

Robustness, Privacy, and Generalization of Adversarial Training

Fine-grained analysis of non-parametric estimation for pairwise learning

Adversarially Robust Generalization Requires More Data

A PAC-Bayes Analysis of Adversarial Robustness

Bridging the Gap: Rademacher Complexity in Robust and Standard Generalization

Successful Daptomycin Treatment for Staphylococcus lugdunensis Endocarditis

Inter-feature Relationship Certifies Robust Generalization of Adversarial Training

Generalization Properties of Adversarial Training for $\ell_0$-Bounded Adversarial Attacks

A Simplified Heuristic Version of Raviv's Algorithm for Using Context in Text Recognition

Understanding Adversarially Robust Generalization via Weight-Curvature Index

Analysis and Applications of Class-wise Robustness in Adversarial Training

Stability and Generalization in Free Adversarial Training

Data-Dependent Stability Analysis of Adversarial Training

Improving Adversarial Robustness to Sensitivity and Invariance Attacks with Deep Metric Learning