Abstract: The ability to preserve local geometry of highly nonlinear manifolds in high dimensional spaces and properly unfold them into lower dimensional hyperplanes is the key to the success of manifold computing, nonlinear dimensionality reduction (NLDR) and visualization. This paper proposes a novel method, called elastic locally isometric smoothness (ELIS), to empower deep neural networks with such an ability. ELIS requires that a desired metric between points should be preserved across layers in order to preserve local geometry; such a smoothness constraint effectively regularizes vector-based transformations to become well-behaved local metric-preserving homeomorphisms. Moreover, ELIS requires that the smoothness should be imposed in a way to render sufficient flexibility for tackling complicated nonlinearity and non-Euclideanity; this is achieved layer-wisely via nonlinearity in both the similarity and activation functions. The ELIS method incorporates a class of suitable nonlinear similarity functions into a two-way divergence loss and uses hyperparameter continuation in finding optimal solutions. Extensive experiments, comparisons, and ablation study demonstrate that ELIS can deliver results not only superior to UMAP and t-SNE for and visualization but also better than other leading counterparts of manifold and autoencoder learning for NLDR and manifold data generation.

Active learning on manifolds

Learning Exemplar-Represented Manifolds in Latent Space for Classification.

Learning an Image Manifold for Retrieval

Deep Manifold Transformation for Dimension Reduction

Deep Manifold Computing and Visualization

Deep Manifold Computing and Visualization Using Elastic Locally Isometric Smoothness

Manifold Adaptive Experimental Design for Text Categorization

Deep Manifold Transformation for Nonlinear Dimensionality Reduction

Markov-Lipschitz Deep Learning

A Regularized Approach for Geodesic-Based Semisupervised Multimanifold Learning

Tangent Space Learning and Generalization

Geodesic Based Semi-supervised Multi-manifold Feature Extraction

A geometric viewpoint of manifold learning

Learning Manifold Structures With Subspace Segmentations

Generalized Clustering and Multi-Manifold Learning with Geometric Structure Preservation

Deep Active Learning with Manifold-preserving Trajectory Sampling

Active subspace learning

Riemannian manifold learning.

Distance Learner: Incorporating Manifold Prior to Model Training

Active Learning Based on Local Representation.