Abstract:This work provides a computable, direct, and mathematically rigorous approximation to the differential geometry of class manifolds for high-dimensional data, along with nonlinear projections from input space onto these class manifolds. The tools are applied to the setting of neural network image classifiers, where we generate novel, on-manifold data samples, and implement a projected gradient descent algorithm for on-manifold adversarial training. The susceptibility of neural networks (NNs) to adversarial attack highlights the brittle nature of NN decision boundaries in input space. Introducing adversarial examples during training has been shown to reduce the susceptibility of NNs to adversarial attack; however, it has also been shown to reduce the accuracy of the classifier if the examples are not valid examples for that class. Realistic "on-manifold" examples have been previously generated from class manifolds in the latent of an autoencoder. Our work explores these phenomena in a geometric and computational setting that is much closer to the raw, high-dimensional input space than can be provided by VAE or other black box dimensionality reductions. We employ conformally invariant diffusion maps (CIDM) to approximate class manifolds in diffusion coordinates, and develop the Nyström projection to project novel points onto class manifolds in this setting. On top of the manifold approximation, we leverage the spectral exterior calculus (SEC) to determine geometric quantities such as tangent vectors of the manifold. We use these tools to obtain adversarial examples that reside on a class manifold, yet fool a classifier. These misclassifications then become explainable in terms of human-understandable manipulations within the data, by expressing the on-manifold adversary in the semantic basis on the manifold.

Projective Manifold Gradient Layer for Deep Rotation Regression

UniGrad-FS: Unified Gradient Projection with Flatter Sharpness for Continual Learning

Deep Regression on Manifolds: A 3D Rotation Case Study

Dynamically Stable Poincaré Embeddings for Neural Manifolds

Deep Manifold Computing and Visualization Using Elastic Locally Isometric Smoothness

Deep Manifold Computing and Visualization

Deep Manifold Transformation for Nonlinear Dimensionality Reduction

Markov-Lipschitz Deep Learning

RMLR: Extending Multinomial Logistic Regression into General Geometries

Learning to Optimize on Riemannian Manifolds

Matrix Manifold Neural Networks++

Deep Projective Rotation Estimation through Relative Supervision

Supervised sparse manifold regression for head pose estimation in 3D space.

Residual Deep Gaussian Processes on Manifolds

Learning Pose Image Manifolds Using Geometry-Preserving GANs and Elasticae

Deep Extrinsic Manifold Representation for Vision Tasks

ManifoldNet: A Deep Network Framework for Manifold-valued Data

Building Neural Networks on Matrix Manifolds: A Gyrovector Space Approach

Non-parametric regression for robot learning on manifolds

On-Manifold Projected Gradient Descent

Manifold Optimization Assisted Gaussian Variational Approximation