Abstract:A fundamental task in data exploration is to extract simplified low dimensional representations that capture intrinsic geometry in data, especially for faithfully visualizing data in two or three dimensions. Common approaches to this task use kernel methods for manifold learning. However, these methods typically only provide an embedding of fixed input data and cannot extend to new data points. Autoencoders have also recently become popular for representation learning. But while they naturally compute feature extractors that are both extendable to new data and invertible (i.e., reconstructing original features from latent representation), they have limited capabilities to follow global intrinsic geometry compared to kernel-based manifold learning. We present a new method for integrating both approaches by incorporating a geometric regularization term in the bottleneck of the autoencoder. Our regularization, based on the diffusion potential distances from the recently-proposed PHATE visualization method, encourages the learned latent representation to follow intrinsic data geometry, similar to manifold learning algorithms, while still enabling faithful extension to new data and reconstruction of data in the original feature space from latent coordinates. We compare our approach with leading kernel methods and autoencoder models for manifold learning to provide qualitative and quantitative evidence of our advantages in preserving intrinsic structure, out of sample extension, and reconstruction. Our method is easily implemented for big-data applications, whereas other methods are limited in this regard.

Extensions of Manifold Learning Algorithms in Kernel Feature Space

Learning Exemplar-Represented Manifolds in Latent Space for Classification.

A Novel Image Classification Method Based on Manifold Learning and Gaussian Mixture Model

Deep Manifold Computing and Visualization

Nonlinear Process Monitoring Using Improved Kernel Principal Component Analysis

IKLTSA: An Incremental Kernel LTSA Method

Learning Manifold Structures With Subspace Segmentations

Investigating Manifold Learning Algorithms Based on Magnification Factors and Principal Spread Directions

Learning a Data-Dependent Kernel Function for KPCA-based Nonlinear Process Monitoring

Embedding New Data Points for Manifold Learning Via Coordinate Propagation.

Adaptive kernel subspace method for speeding up feature extraction

On Feature Extraction Via Kernels

Local Smoothing For Manifold Learning

Synthesis and characterization of N,N'-carbonyl difatty amides from palm oil.

Multilinear Kernel Regression and Imputation via Manifold Learning

Clustering analysis using manifold kernel concept factorization.

Patterns classification of nonlinear multi-dimensional time series based on manifold learning

Manifold Analysis In Reconstructed State Space For Nonlinear Signal Classification

Non-parametric regression for robot learning on manifolds

Extendable and invertible manifold learning with geometry regularized autoencoders

Feature Expansion and Feature Selection for General Pattern Recognition Problems