Abstract:Autoencoding is a popular method in representation learning. Conventional autoencoders employ symmetric encoding-decoding procedures and a simple Euclidean latent space to detect hidden low-dimensional structures in an unsupervised way. Some modern approaches to novel data generation such as generative adversarial networks askew this symmetry, but still employ a pair of massive networks--one to generate the image and another to judge the images quality based on priors learned from a training set. This work introduces a chart autoencoder with an asymmetric encoding-decoding process that can incorporate additional semi-supervised information such as class labels. Besides enhancing the capability for handling data with complicated topological and geometric structures, the proposed model can successfully differentiate nearby but disjoint manifolds and intersecting manifolds with only a small amount of supervision. Moreover, this model only requires a low-complexity encoding operation, such as a locally defined linear projection. We discuss the approximation power of such networks and derive a bound that essentially depends on the intrinsic dimension of the data manifold rather than the dimension of ambient space. Next we incorporate bounds for the sampling rate of training data need to faithfully represent a given data manifold. We present numerical experiments that verify that the proposed model can effectively manage data with multi-class nearby but disjoint manifolds of different classes, overlapping manifolds, and manifolds with non-trivial topology. Finally, we conclude with some experiments on computer vision and molecular dynamics problems which showcase the efficacy of our methods on real-world data.

Deep nonparametric estimation of intrinsic data structures by chart autoencoders: Generalization error and robustness

Deep Nonparametric Estimation of Intrinsic Data Structures by Chart Autoencoders: Generalization Error and Robustness

Learning Robust Features with Incremental Auto-Encoders

Chart Auto-Encoders for Manifold Structured Data

Semi-Supervised Manifold Learning with Complexity Decoupled Chart Autoencoders

Autoencoders for discovering manifold dimension and coordinates in data from complex dynamical systems

A Label Noise Robust Stacked Auto-Encoder Algorithm for Inaccurate Supervised Classification Problems

A Priori Estimates of the Generalization Error for Autoencoders.

Generalized Autoencoder: A Neural Network Framework for Dimensionality Reduction

High-dimensional Asymptotics of Denoising Autoencoders

The dynamics of representation learning in shallow, non-linear autoencoders

Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depth

Analyzing noise in autoencoders and deep networks

High-dimensional asymptotics of denoising autoencoders *

On a Mechanism Framework of Autoencoders

Thinner Latent Spaces: Detecting dimension and imposing invariance through autoencoder gradient constraints

Why should autoencoders work?

Dimensionality Reduction Strategy Based on Auto-Encoder

Stacked autoencoders based machine learning for noise reduction and signal reconstruction in geophysical data

Application of graph auto-encoders based on regularization in recommendation algorithms

An Automated Data Mining Framework Using Autoencoders for Feature Extraction and Dimensionality Reduction