Abstract:Deep neural networks are known to be vulnerable to adversarial attacks (AA). For an image recognition task, this means that a small perturbation of the original can result in the image being misclassified. Design of such attacks as well as methods of adversarial training against them are subject of intense research. We re-cast the problem using techniques of Wasserstein distributionally robust optimization (DRO) and obtain novel contributions leveraging recent insights from DRO sensitivity analysis. We consider a set of distributional threat models. Unlike the traditional pointwise attacks, which assume a uniform bound on perturbation of each input data point, distributional threat models allow attackers to perturb inputs in a non-uniform way. We link these more general attacks with questions of out-of-sample performance and Knightian uncertainty. To evaluate the distributional robustness of neural networks, we propose a first-order AA algorithm and its multi-step version. Our attack algorithms include Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) as special cases. Furthermore, we provide a new asymptotic estimate of the adversarial accuracy against distributional threat models. The bound is fast to compute and first-order accurate, offering new insights even for the pointwise AA. It also naturally yields out-of-sample performance guarantees. We conduct numerical experiments on the CIFAR-10 dataset using DNNs on RobustBench to illustrate our theoretical results. Our code is available at <a class="link-external link-https" href="https://github.com/JanObloj/W-DRO-Adversarial-Methods" rel="external noopener nofollow">this https URL</a>.

Robust Estimation under the Wasserstein Distance

Robust Distribution Learning with Local and Global Adversarial Corruptions

Robust W-GAN-Based Estimation Under Wasserstein Contamination

A Robust Learning Algorithm for Regression Models Using Distributionally Robust Optimization under the Wasserstein Metric

Outlier-Robust Wasserstein DRO

Confidence regions in Wasserstein distributionally robust estimation

Robust Hypothesis Testing Using Wasserstein Uncertainty Sets

Robust Hypothesis Testing with Wasserstein Uncertainty Sets

On Projection Robust Optimal Transport: Sample Complexity and Model Misspecification

Universal Robust Regression via Maximum Mean Discrepancy

Robustified Multivariate Regression and Classification Using Distributionally Robust Optimization under the Wasserstein Metric

Inference via robust optimal transportation: theory and methods

Unifying Distributionally Robust Optimization via Optimal Transport Theory

Wasserstein Distributionally Robust Optimization with Heterogeneous Data Sources

Regularization for Wasserstein distributionally robust optimization

Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning

Quantifying Distributional Model Risk in Marginal Problems via Optimal Transport

Distributionally Robust Prescriptive Analytics with Wasserstein Distance

Wasserstein distributional robustness of neural networks

Outlier detection using distributionally robust optimization under the Wasserstein metric

A Robust Learning Approach for Regression Models Based on Distributionally Robust Optimization.