Abstract:Deep learning models are known to be vulnerable to adversarial attacks. Adversarial learning is therefore becoming a crucial task. We propose a new vision on neural network robustness using Riemannian geometry and foliation theory. The idea is illustrated by creating a new adversarial attack that takes into account the curvature of the data space. This new adversarial attack, called the two-step spectral attack , is a piece-wise linear approximation of a geodesic in the data space. The data space is treated as a (degenerate) Riemannian manifold equipped with the pullback of the Fisher Information Metric (FIM) of the neural network. In most cases, this metric is only semi-definite and its kernel becomes a central object to study. A canonical foliation is derived from this kernel. The curvature of transverse leaves gives the appropriate correction to get a two-step approximation of the geodesic and hence a new efficient adversarial attack. The method is first illustrated on a 2D toy example in order to visualize the neural network foliation and the corresponding attacks. Next, we report numerical results on the MNIST and CIFAR10 datasets with the proposed technique and state of the art attacks presented by Zhao et al. (in: Proceedings of the AAAI conference on artificial intelligence, vol 33. pp 5869–5876, 2019) (OSSA) and Croce and Hein (in: III HD, Singh A (eds) Proceedings of machine learning research, vol 119, PMLR, Cambridge, pp 2206–2216, https://proceedings.mlr.press/v119/croce20b.html, 2020) (AutoAttack). The results show that the proposed attack is more efficient at all levels of available budget for the attack (norm of the attack), confirming that the curvature of the transverse neural network FIM foliation plays an important role in the robustness of neural networks. The main objective and interest of this study is to provide a mathematical understanding of the geometrical issues at play in the data space when constructing efficient attacks on neural networks.

Adversarial attacks on neural networks through canonical Riemannian foliations

Adversarial attacks on neural networks through canonical Riemannian foliations

Attack As Defense: Characterizing Adversarial Examples Using Robustness.

NeRFail: Neural Radiance Fields-Based Multiview Adversarial Attack

A Geometric Framework for Adversarial Vulnerability in Machine Learning

Improving the Robustness of Adversarial Attacks Using an Affine-Invariant Gradient Estimator

Not So Robust After All: Evaluating the Robustness of Deep Neural Networks to Unseen Adversarial Attacks

Deviations in Representations Induced by Adversarial Attacks

Trust Region Based Adversarial Attack on Neural Networks

Hierarchical binding in convolutional neural networks: Making adversarial attacks geometrically challenging

On-Manifold Projected Gradient Descent

Understanding Adversarial Robustness Via Critical Attacking Route.

Adversarial Attack and Interpretability of the Deep Neural Net-Work from the Geometric Perspective

Compositional Curvature Bounds for Deep Neural Networks

Towards the first adversarially robust neural network model on MNIST

The Adversarial Attack and Detection under the Fisher Information Metric

Bio-Inspired Adversarial Attack Against Deep Neural Networks

Defense Against Adversarial Attacks via Controlling Gradient Leaking on Embedded Manifolds

Towards Deep Learning Models Resistant to Adversarial Attacks

Robustness of 3D Deep Learning in an Adversarial Setting

An Empirical Study on the Relation between Network Interpretability and Adversarial Robustness