Abstract:Dimensionality reduction has many applications in data visualization and machine learning. Existing methods can be classified into global ones and local ones. The global methods usually learn the linear relationship in data, while the local ones learn the manifold intrinsic geometry structure, which has a significant impact on pattern recognition. However, most of existing local methods obtain an embedding with eigenvalue or singular value decomposition, where the computational complexities are very high in a large amount of high-dimensional data. In this paper, we propose a local nonlinear dimensionality reduction method named Vec2vec , which employs a neural network with only one hidden layer to reduce the computational complexity. We first build a neighborhood similarity graph from the input matrix, and then define the context of data points with the random walk properties in the graph. Finally, we train the neural network with the context of data points to learn the embedding of the matrix. We conduct extensive experiments of data classification and clustering on nine image and text datasets to evaluate the performance of our method. Experimental results show that Vec2vec is better than several state-of-the-art dimensionality reduction methods, except that it is equivalent to UMAP on data clustering tasks in the statistical hypothesis tests, but Vec2vec needs less computational time than UMAP in high-dimensional data. Furthermore, we propose a more lightweight method named Approximate Vec2vec (AVec2vec) with little performance degradation, which employs an approximate method to build the neighborhood similarity graph. AVec2vec is still better than some state-of-the-art local dimensionality reduction methods and competitive with UMAP on data classification and clustering tasks in the statistical hypothesis tests.

Structure-Preserving Non-Linear PCA for Matrices

Structure-preserving non-linear PCA for matrices

Matrix normal PCA for interpretable dimension reduction and graphical noise modeling

Maximally Correlated Principal Component Analysis

Linear Dimensionality Reduction Based on Hybrid Structure Preserving Projections

Locality Preserving Nonnegative Matrix Factorization.

Nonlinear Dimensionality Reduction for Discriminative Analytics of Multiple Datasets

A Local Similarity-Preserving Framework for Nonlinear Dimensionality Reduction with Neural Networks

@ An Overview for Non-Negative Matrix Factorization

$σ$-PCA: a building block for neural learning of identifiable linear transformations

A Locality Preserving Approach for Kernel PCA

Avoiding Optimal Mean Robust PCA/2DPCA with Non-Greedy L1-norm Maximization

nPCA: a linear dimensionality reduction method using a multilayer perceptron

Manifold Alignment Via Global and Local Structures Preserving PCA Framework.

Distortion-Free Nonlinear Dimensionality Reduction

Recovering Hidden Diagonal Structures Via Non-Negative Matrix Factorization with Multiple Constraints

Local Nonlinear Dimensionality Reduction via Preserving the Geometric Structure of Data

Nonlinear and Nonparametric Extensions

Beyond Low Rank + Sparse: Multiscale Low Rank Matrix Decomposition

A Second-Order Mapping PCA Based Intrinsic Dimension Estimate for Nonlinear Time Series

Neighborhood Preserving Nonnegative Matrix Factorization