Abstract:Research on improving the robustness of neural networks to adversarial noise - imperceptible malicious perturbations of the data - has received significant attention. The currently uncontested state-of-the-art defense to obtain robust deep neural networks is Adversarial Training (AT), but it consumes significantly more resources compared to standard training and trades off accuracy for robustness. An inspiring recent work [Dapello et al.] aims to bring neurobiological tools to the question: How can we develop Neural Nets that robustly generalize like human vision? [Dapello et al.] design a network structure with a neural hidden first layer that mimics the primate primary visual cortex (V1), followed by a back-end structure adapted from current CNN vision models. It seems to achieve non-trivial adversarial robustness on standard vision benchmarks when tested on small perturbations. Here we revisit this biologically inspired work, and ask whether a principled parameter-free representation with inspiration from physics is able to achieve the same goal. We discover that the wavelet scattering transform can replace the complex V1-cortex and simple uniform Gaussian noise can take the role of neural stochasticity, to achieve adversarial robustness. In extensive experiments on the CIFAR-10 benchmark with adaptive adversarial attacks we show that: 1) Robustness of VOneBlock architectures is relatively weak (though non-zero) when the strength of the adversarial attack radius is set to commonly used benchmarks. 2) Replacing the front-end VOneBlock by an off-the-shelf parameter-free Scatternet followed by simple uniform Gaussian noise can achieve much more substantial adversarial robustness without adversarial training. Our work shows how physically inspired structures yield new insights into robustness that were previously only thought possible by meticulously mimicking the human cortex.

Fixed Inter-Neuron Covariability Induces Adversarial Robustness

Improving Model Robustness Against Adversarial Examples with Redundant Fully Connected Layer.

Improving the Robustness of Deep Convolutional Neural Networks Through Feature Learning

Adversarial alignment: Breaking the trade-off between the strength of an attack and its relevance to human perception

Adversarial robustness improvement for deep neural networks

Leveraging the Human Ventral Visual Stream to Improve Neural Network Robustness

Progressive Diversified Augmentation for General Robustness of DNNs: A Unified Approach.

Towards Certifying the Asymmetric Robustness for Neural Networks: Quantification and Applications

Interpreting and Improving Adversarial Robustness of Deep Neural Networks With Neuron Sensitivity

Dense Associative Memory is Robust to Adversarial Inputs

Wavelets Beat Monkeys at Adversarial Robustness

ROBUSTNESS OF DEEP NEURAL NETWORKS TO ADVERSARIAL EXAMPLES

On Inherent Adversarial Robustness of Active Vision Systems

Not So Robust After All: Evaluating the Robustness of Deep Neural Networks to Unseen Adversarial Attacks

Local Competition and Uncertainty for Adversarial Robustness in Deep Learning

Bio-inspired Robustness: A Review

Generalization ability and Vulnerabilities to adversarial perturbations: Two sides of the same coin

A Regularization Perspective Based Theoretical Analysis for Adversarial Robustness of Deep Spiking Neural Networks.

Layerwise Hebbian/anti-Hebbian (HaH) Learning In Deep Networks: A Neuro-inspired Approach To Robustness

Towards Improving Robustness Against Common Corruptions in Object Detectors Using Adversarial Contrastive Learning

SNN-RAT: Robustness-enhanced Spiking Neural Network through Regularized Adversarial Training