Abstract: The ability to preserve local geometry of highly nonlinear manifolds in high dimensional spaces and properly unfold them into lower dimensional hyperplanes is the key to the success of manifold computing, nonlinear dimensionality reduction (NLDR) and visualization. This paper proposes a novel method, called elastic locally isometric smoothness (ELIS), to empower deep neural networks with such an ability. ELIS requires that a desired metric between points should be preserved across layers in order to preserve local geometry; such a smoothness constraint effectively regularizes vector-based transformations to become well-behaved local metric-preserving homeomorphisms. Moreover, ELIS requires that the smoothness should be imposed in a way to render sufficient flexibility for tackling complicated nonlinearity and non-Euclideanity; this is achieved layer-wisely via nonlinearity in both the similarity and activation functions. The ELIS method incorporates a class of suitable nonlinear similarity functions into a two-way divergence loss and uses hyperparameter continuation in finding optimal solutions. Extensive experiments, comparisons, and ablation study demonstrate that ELIS can deliver results not only superior to UMAP and t-SNE for and visualization but also better than other leading counterparts of manifold and autoencoder learning for NLDR and manifold data generation.

DIMAL: Deep Isometric Manifold Learning Using Sparse Geodesic Sampling

Deep Manifold Computing and Visualization

Deep Manifold Transformation for Dimension Reduction

Deep Manifold Computing and Visualization Using Elastic Locally Isometric Smoothness

Deep Manifold Transformation for Nonlinear Dimensionality Reduction

Markov-Lipschitz Deep Learning

A Regularized Approach for Geodesic-Based Semisupervised Multimanifold Learning

Geodesic Based Semi-supervised Multi-manifold Feature Extraction

Inductive Global and Local Manifold Approximation and Projection

Isometric Immersion Learning with Riemannian Geometry

Efficient Representation of Low-Dimensional Manifolds using Deep Networks

Geometric Deep Learning: Going beyond Euclidean data

Distance Penalization Embedding for Unsupervised Dimensionality Reduction

Active learning on manifolds

DLME: Deep Local-flatness Manifold Embedding

Grassmannian Learning: Embedding Geometry Awareness in Shallow and Deep Learning

DeepPoint3D: Learning discriminative local descriptors using deep metric learning on 3D point clouds

Deep Spatiality: Unsupervised Learning of Spatially-Enhanced Global and Local 3D Features by Deep Neural Network With Coupled Softmax.

DeepMIF: Deep Monotonic Implicit Fields for Large-Scale LiDAR 3D Mapping

Geodesic Learning with Uniform Interpolation on Data Manifold