Abstract:Successful deep learning models often involve training neural network architectures that contain more parameters than the number of training samples. Such overparametrized models have been extensively studied in recent years, and the virtues of overparametrization have been established from both the statistical perspective, via the double-descent phenomenon, and the computational perspective via the structural properties of the optimization landscape. Despite the remarkable success of deep learning architectures in the overparametrized regime, it is also well known that these models are highly vulnerable to small adversarial perturbations in their inputs. Even when adversarially trained, their performance on perturbed inputs (robust generalization) is considerably worse than their best attainable performance on benign inputs (standard generalization). It is thus imperative to understand how overparametrization fundamentally affects robustness. In this paper, we will provide a precise characterization of the role of overparametrization on robustness by focusing on random features regression models (two-layer neural networks with random first layer weights). We consider a regime where the sample size, the input dimension and the number of parameters grow in proportion to each other, and derive an asymptotically exact formula for the robust generalization error when the model is adversarially trained. Our developed theory reveals the nontrivial effect of overparametrization on robustness and indicates that for adversarially trained random features models, high overparametrization can hurt robust generalization.

Over-parameterization and Adversarial Robustness in Neural Networks: An Overview and Empirical Analysis

Attack As Defense: Characterizing Adversarial Examples Using Robustness.

Convergence of Adversarial Training in Overparametrized Neural Networks

Feature Augmentation for Adversarial Robustness

Are Adversarial Robustness and Common Perturbation Robustness Independent Attributes ?

ROBUSTNESS OF DEEP NEURAL NETWORKS TO ADVERSARIAL EXAMPLES

The curse of overparametrization in adversarial training: Precise analysis of robust generalization for random features regression

Improving Model Robustness Against Adversarial Examples with Redundant Fully Connected Layer.

Adversarial Robustness of Deep Neural Networks: A Survey from a Formal Verification Perspective

L G ] 1 9 Ju n 20 19 Convergence of Adversarial Training in Overparametrized Networks

The Surprising Harmfulness of Benign Overfitting for Adversarial Robustness

Interpreting and Evaluating Neural Network Robustness

Towards Class-wise Robustness Analysis

Towards Robust Training of Neural Networks by Regularizing Adversarial Gradients

Measuring Neural Net Robustness with Constraints

Interpreting and Improving Adversarial Robustness of Deep Neural Networks With Neuron Sensitivity

When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks

Towards Adversarial Robustness with Multidimensional Perturbations Via Contrastive Learning

Not So Robust After All: Evaluating the Robustness of Deep Neural Networks to Unseen Adversarial Attacks

Robustness of Neural Networks: A Probabilistic and Practical Approach

Towards Evaluating the Robustness of Neural Networks