Abstract: The ability to preserve local geometry of highly nonlinear manifolds in high dimensional spaces and properly unfold them into lower dimensional hyperplanes is the key to the success of manifold computing, nonlinear dimensionality reduction (NLDR) and visualization. This paper proposes a novel method, called elastic locally isometric smoothness (ELIS), to empower deep neural networks with such an ability. ELIS requires that a desired metric between points should be preserved across layers in order to preserve local geometry; such a smoothness constraint effectively regularizes vector-based transformations to become well-behaved local metric-preserving homeomorphisms. Moreover, ELIS requires that the smoothness should be imposed in a way to render sufficient flexibility for tackling complicated nonlinearity and non-Euclideanity; this is achieved layer-wisely via nonlinearity in both the similarity and activation functions. The ELIS method incorporates a class of suitable nonlinear similarity functions into a two-way divergence loss and uses hyperparameter continuation in finding optimal solutions. Extensive experiments, comparisons, and ablation study demonstrate that ELIS can deliver results not only superior to UMAP and t-SNE for and visualization but also better than other leading counterparts of manifold and autoencoder learning for NLDR and manifold data generation.

Study on the Low-Dimensional Embedding and the Embedding Dimensionality of Manifold of High-Dimensional Data

Deep Manifold Transformation for Dimension Reduction

Deep Manifold Computing and Visualization

Deep Manifold Transformation for Nonlinear Dimensionality Reduction

Deep Manifold Computing and Visualization Using Elastic Locally Isometric Smoothness

A New Manifold Reconstruction Method Based on Isomap

Investigating Manifold Learning Algorithms Based on Magnification Factors and Principal Spread Directions

Nonlinear Dimensionality Reduction by Locally Linear Inlaying

Isometric Multi-manifold Learning for Feature Extraction

A New Approach to Discover Interlacing Data Structures in High-Dimensional Space

Distance Penalization Embedding for Unsupervised Dimensionality Reduction

CN-Isomap Algorithm for Nonlinear Dimensionality Reduction of Sparse Data

The connections between principal component analysis and dimensionality reduction methods of manifolds

Overview of Manifold Learning

Invertible Manifold Learning for Dimension Reduction

Using ISOMAP to extract information from satellite ocean temperature images

Isometric Embedding and Continuum ISOMAP.

A Method of Nonlinear Manifold Classification Using ISOMAP

Manifold Learning for Visualizing and Analyzing High-Dimensional Data

Adaptive Nonlinear Dimensionality Reduction with a Local Metric.

Manifold-based learning and synthesis.