Abstract: The ability to preserve local geometry of highly nonlinear manifolds in high dimensional spaces and properly unfold them into lower dimensional hyperplanes is the key to the success of manifold computing, nonlinear dimensionality reduction (NLDR) and visualization. This paper proposes a novel method, called elastic locally isometric smoothness (ELIS), to empower deep neural networks with such an ability. ELIS requires that a desired metric between points should be preserved across layers in order to preserve local geometry; such a smoothness constraint effectively regularizes vector-based transformations to become well-behaved local metric-preserving homeomorphisms. Moreover, ELIS requires that the smoothness should be imposed in a way to render sufficient flexibility for tackling complicated nonlinearity and non-Euclideanity; this is achieved layer-wisely via nonlinearity in both the similarity and activation functions. The ELIS method incorporates a class of suitable nonlinear similarity functions into a two-way divergence loss and uses hyperparameter continuation in finding optimal solutions. Extensive experiments, comparisons, and ablation study demonstrate that ELIS can deliver results not only superior to UMAP and t-SNE for and visualization but also better than other leading counterparts of manifold and autoencoder learning for NLDR and manifold data generation.

Manifold Learning using Growing Locally Linear Embedding

Growing Locally Linear Embedding for Manifold Learning

Deep Manifold Transformation for Dimension Reduction

Deep Manifold Transformation for Nonlinear Dimensionality Reduction

Deep Manifold Computing and Visualization Using Elastic Locally Isometric Smoothness

Deep Manifold Computing and Visualization

Markov-Lipschitz Deep Learning

Local Neighbor Propagation Embedding.

Generalized Locally Linear Embedding Based on Local Reconstruction Similarity

Noisy Manifold Learning Using Neighborhood Smoothing Embedding

Nonlinear Dimensionality Reduction by Locally Linear Inlaying

Integrating Local and Global Manifold Structures for Unsupervised Dimensionality Reduction

Local Linear Embedding with Adaptive Neighbors.

Fusion of local and globle structures for manifold learning

Inductive Global and Local Manifold Approximation and Projection

DLME: Deep Local-flatness Manifold Embedding

Nonlinear embedding preserving multiple local-linearities

Manifold Learning in Local Tangent Space Via Extreme Learning Machine

AN IMPROVED ALGORITHM ON LOCALLY LINEAR EMBEDDING

Hierarchical Neighbors Embedding