Abstract:Two key challenges facing modern deep learning are mitigating deep networks' vulnerability to adversarial attacks and understanding deep learning's generalization capabilities. Towards the first issue, many defense strategies have been developed, with the most common being Adversarial Training (AT). Towards the second challenge, one of the dominant theories that has emerged is the Neural Tangent Kernel (NTK) -- a characterization of neural network behavior in the infinite-width limit. In this limit, the kernel is frozen, and the underlying feature map is fixed. In finite widths, however, there is evidence that feature learning happens at the earlier stages of the training (kernel learning) before a second phase where the kernel remains fixed (lazy training). While prior work has aimed at studying adversarial vulnerability through the lens of the frozen infinite-width NTK, there is no work that studies the adversarial robustness of the empirical/finite NTK during training. In this work, we perform an empirical study of the evolution of the empirical NTK under standard and adversarial training, aiming to disambiguate the effect of adversarial training on kernel learning and lazy training. We find under adversarial training, the empirical NTK rapidly converges to a different kernel (and feature map) than standard training. This new kernel provides adversarial robustness, even when non-robust training is performed on top of it. Furthermore, we find that adversarial training on top of a fixed kernel can yield a classifier with $76.1\%$ robust accuracy under PGD attacks with $\varepsilon = 4/255$ on CIFAR-10.

L G ] 1 9 Ju n 20 19 Convergence of Adversarial Training in Overparametrized Networks

Convergence of Adversarial Training in Overparametrized Neural Networks

Towards Robust DNNs: an Taylor Expansion-Based Method for Generating Powerful Adversarial Examples.

Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality

Over-parameterization and Adversarial Robustness in Neural Networks: An Overview and Empirical Analysis

Adversarial Training: embedding adversarial perturbations into the parameter space of a neural network to build a robust system

Improving Adversarial Robustness of Deep Neural Networks Via Linear Programming

Towards Adversarial Robustness with Multidimensional Perturbations Via Contrastive Learning

Towards Robust Training of Neural Networks by Regularizing Adversarial Gradients

Evolution of Neural Tangent Kernels under Benign and Adversarial Training

You Only Propagate Once: Painless Adversarial Training Using Maximal Principle

A Direct Approach to Robust Deep Learning Using Adversarial Networks

Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization

A constrained optimization approach to improve robustness of neural networks

Adversarial Training of Two-Layer Polynomial and ReLU Activation Networks via Convex Optimization

ROBUSTNESS OF DEEP NEURAL NETWORKS TO ADVERSARIAL EXAMPLES

You Only Propagate Once: Accelerating Adversarial Training Via Maximal Principle

Practical Convex Formulation of Robust One-hidden-layer Neural Network Training

Adversarial Robustness Vs. Model Compression, or Both?

Adversarial Transformation Network with Adaptive Perturbations for Generating Adversarial Examples.