Abstract:Empirical adversarial risk minimization (EARM) is a widely used mathematical framework to robustly train deep neural nets (DNNs) that are resistant to adversarial attacks. However, both natural and robust accuracies, in classifying clean and adversarial images, respectively, of the trained robust models are far from satisfactory. In this work, we unify the theory of optimal control of transport equations with the practice of training and testing of ResNets. Based on this unified viewpoint, we propose a simple yet effective ResNets ensemble algorithm to boost the accuracy of the robustly trained model on both clean and adversarial images. The proposed algorithm consists of two components: First, we modify the base ResNets by injecting a variance specified Gaussian noise to the output of each residual mapping. Second, we average over the production of multiple jointly trained modified ResNets to get the final prediction. These two steps give an approximation to the Feynman-Kac formula for representing the solution of a transport equation with viscosity, or a convection-diffusion equation. For the CIFAR10 benchmark, this simple algorithm leads to a robust model with a natural accuracy of {\bf 85.62}\% on clean images and a robust accuracy of ${\bf 57.94 \%}$ under the 20 iterations of the IFGSM attack, which outperforms the current state-of-the-art in defending against IFGSM attack on the CIFAR10. Both natural and robust accuracies of the proposed ResNets ensemble can be improved dynamically as the building block ResNet advances. The code is available at: \url{<a class="link-external link-https" href="https://github.com/BaoWangMath/EnResNet" rel="external noopener nofollow">this https URL</a>}.

EnResNet: ResNets Ensemble Via the Feynman-Kac Formalism for Adversarial Defense and Beyond

ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies

An Integrated Approach to Produce Robust Models with High Efficiency

Ensemble-in-One: Ensemble Learning Within Random Gated Networks for Enhanced Adversarial Robustness.

LDN-RC: a Lightweight Denoising Network with Residual Connection to Improve Adversarial Robustness

Sparsity Meets Robustness: Channel Pruning for the Feynman-Kac Formalism Principled Robust Deep Neural Nets

Adversarial Defense Via Self-Orthogonal Randomization Super-Network.

Revisiting Ensembles in an Adversarial Context: Improving Natural Accuracy

Guardian of the Ensembles: Introducing Pairwise Adversarially Robust Loss for Resisting Adversarial Attacks in DNN Ensembles

Understanding and Improving Ensemble Adversarial Defense

When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks

Improving Generalization of Adversarial Training via Robust Critical Fine-Tuning

Voting based ensemble improves robustness of defensive models

Unifying Bilateral Filtering and Adversarial Training for Robust Neural Networks

Towards Robust Training of Neural Networks by Regularizing Adversarial Gradients

Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networks

Towards Robust Deep Learning with Ensemble Networks and Noisy Layers

MixedNUTS: Training-Free Accuracy-Robustness Balance via Nonlinearly Mixed Classifiers

Improved Robustness Against Adaptive Attacks With Ensembles and Error-Correcting Output Codes

On the Certified Robustness for Ensemble Models and Beyond

A constrained optimization approach to improve robustness of neural networks