Abstract:Research on improving the robustness of neural networks to adversarial noise - imperceptible malicious perturbations of the data - has received significant attention. The currently uncontested state-of-the-art defense to obtain robust deep neural networks is Adversarial Training (AT), but it consumes significantly more resources compared to standard training and trades off accuracy for robustness. An inspiring recent work [Dapello et al.] aims to bring neurobiological tools to the question: How can we develop Neural Nets that robustly generalize like human vision? [Dapello et al.] design a network structure with a neural hidden first layer that mimics the primate primary visual cortex (V1), followed by a back-end structure adapted from current CNN vision models. It seems to achieve non-trivial adversarial robustness on standard vision benchmarks when tested on small perturbations. Here we revisit this biologically inspired work, and ask whether a principled parameter-free representation with inspiration from physics is able to achieve the same goal. We discover that the wavelet scattering transform can replace the complex V1-cortex and simple uniform Gaussian noise can take the role of neural stochasticity, to achieve adversarial robustness. In extensive experiments on the CIFAR-10 benchmark with adaptive adversarial attacks we show that: 1) Robustness of VOneBlock architectures is relatively weak (though non-zero) when the strength of the adversarial attack radius is set to commonly used benchmarks. 2) Replacing the front-end VOneBlock by an off-the-shelf parameter-free Scatternet followed by simple uniform Gaussian noise can achieve much more substantial adversarial robustness without adversarial training. Our work shows how physically inspired structures yield new insights into robustness that were previously only thought possible by meticulously mimicking the human cortex.

Analysis of Random Perturbations for Robust Convolutional Neural Networks

Constant Random Perturbations Provide Adversarial Robustness with Minimal Effect on Accuracy

An Empirical Study on the Effect of Training Data Perturbations on Neural Network Robustness

Are Adversarial Robustness and Common Perturbation Robustness Independent Attributes ?

Understanding the Robustness of Randomized Feature Defense Against Query-Based Adversarial Attacks

Adversarial Training and Robustness for Multiple Perturbations

RANDOM MASK: Towards Robust Convolutional Neural Networks

Fight Perturbations with Perturbations: Defending Adversarial Attacks via Neuron Influence

An Empirical Investigation of Randomized Defenses against Adversarial Attacks

Classifying Perturbation Types for Robustness Against Multiple Adversarial Perturbations

Analyzing the Noise Robustness of Deep Neural Networks

A Survey of Neural Network Robustness Assessment in Image Recognition

Towards the first adversarially robust neural network model on MNIST

Understanding Adversarial Robustness from Feature Maps of Convolutional Layers

Architectural Resilience to Foreground-and-Background Adversarial Noise

Wavelets Beat Monkeys at Adversarial Robustness

Towards Robustness against Unsuspicious Adversarial Examples

Generating Universal Adversarial Perturbation with ResNet

Attacking Adversarial Attacks as A Defense

Adversarial Robustness of Deep Neural Networks: A Survey from a Formal Verification Perspective