Abstract:Deep neural networks (DNNs) have had many successes, but they suffer from two major issues: (1) a vulnerability to adversarial examples and (2) a tendency to elude human interpretation. Interestingly, recent empirical and theoretical evidence suggests these two seemingly disparate issues are actually connected. In particular, robust models tend to provide more interpretable gradients than non-robust models. However, whether this relationship works in the opposite direction remains obscure. With this paper, we seek empirical answers to the following question: can models acquire adversarial robustness when they are trained to have interpretable gradients? We introduce a theoretically inspired technique called Interpretation Regularization (IR), which encourages a model's gradients to (1) match the direction of interpretable target salience maps and (2) have small magnitude. To assess model performance and tease apart factors that contribute to adversarial robustness, we conduct extensive experiments on MNIST and CIFAR-10 with both $\ell_2$ and $\ell_\infty$ attacks. We demonstrate that training the networks to have interpretable gradients improves their robustness to adversarial perturbations. Applying the network interpretation technique SmoothGrad yields additional performance gains, especially in cross-norm attacks and under heavy perturbations. The results indicate that the interpretability of the model gradients is a crucial factor for adversarial robustness. Code for the experiments can be found at <a class="link-external link-https" href="https://github.com/a1noack/interp_regularization" rel="external noopener nofollow">this https URL</a>.

On Connections Between Regularizations for Improving DNN Robustness

Wordreg: Mitigating the Gap Between Training and Inference with Worst-Case Drop Regularization

Regularization for Adversarial Robust Learning

Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing Their Input Gradients

Improving Interpretability via Regularization of Neural Activation Sensitivity

Towards Robust Training of Neural Networks by Regularizing Adversarial Gradients

An Empirical Study on the Relation between Network Interpretability and Adversarial Robustness

A Brain-Inspired Regularizer for Adversarial Robustness

Generalizing and Improving Jacobian and Hessian Regularization

Network as Regularization for Training Deep Neural Networks: Framework, Model and Performance

Interpreting and Improving Adversarial Robustness of Deep Neural Networks With Neuron Sensitivity

Improving Model Robustness Against Adversarial Examples with Redundant Fully Connected Layer.

Characterizing Model Robustness via Natural Input Gradients

A regularization method to improve adversarial robustness of neural networks for ECG signal classification

Data-Driven Lipschitz Continuity: A Cost-Effective Approach to Improve Adversarial Robustness

On the Geometry of Regularization in Adversarial Training: High-Dimensional Asymptotics and Generalization Bounds

Regularizing Deep Networks Using Efficient Layerwise Adversarial Training

Enhancing Adversarial Robustness in SNNs with Sparse Gradients

Improving the Tightness of Convex Relaxation Bounds for Training Certifiably Robust Classifiers

Practical Convex Formulation of Robust One-hidden-layer Neural Network Training

Robust Sparse Regularization: Simultaneously Optimizing Neural Network Robustness and Compactness