Abstract:Dimensionality reduction has many applications in data visualization and machine learning. Existing methods can be classified into global ones and local ones. The global methods usually learn the linear relationship in data, while the local ones learn the manifold intrinsic geometry structure, which has a significant impact on pattern recognition. However, most of existing local methods obtain an embedding with eigenvalue or singular value decomposition, where the computational complexities are very high in a large amount of high-dimensional data. In this paper, we propose a local nonlinear dimensionality reduction method named Vec2vec , which employs a neural network with only one hidden layer to reduce the computational complexity. We first build a neighborhood similarity graph from the input matrix, and then define the context of data points with the random walk properties in the graph. Finally, we train the neural network with the context of data points to learn the embedding of the matrix. We conduct extensive experiments of data classification and clustering on nine image and text datasets to evaluate the performance of our method. Experimental results show that Vec2vec is better than several state-of-the-art dimensionality reduction methods, except that it is equivalent to UMAP on data clustering tasks in the statistical hypothesis tests, but Vec2vec needs less computational time than UMAP in high-dimensional data. Furthermore, we propose a more lightweight method named Approximate Vec2vec (AVec2vec) with little performance degradation, which employs an approximate method to build the neighborhood similarity graph. AVec2vec is still better than some state-of-the-art local dimensionality reduction methods and competitive with UMAP on data classification and clustering tasks in the statistical hypothesis tests.

Linear Low-Rank Approximation and Nonlinear Dimensionality Reduction

Deep Manifold Transformation for Dimension Reduction

Computational and Theoretical Analysis of Supervised Dimensionality Reduction

Data-driven Dimension Reduction for High-Dimensional Random Vibration Systems with Cubic Nonlinearity

Linear Dimensionality Reduction: Survey, Insights, and Generalizations

Deep Nonlinear Sufficient Dimension Reduction

Real-valued Multivariate Dimension Reduction: A Survey

Riemannian manifold learning for nonlinear dimensionality reduction

Stable Local Dimensionality Reduction Approaches

A Local Similarity-Preserving Framework for Nonlinear Dimensionality Reduction with Neural Networks

Distortion-Free Nonlinear Dimensionality Reduction

Nonlinear Dimension Reduction Via Local Tangent Space Alignment

Dimensionality Reduction Via Subspace and Submanifold Learning [from the Guest Editors]

Robust Structured Nonnegative Matrix Factorization for Image Representation.

Principal manifolds and nonlinear dimension reduction via local tangent space alignment (dept of computer science, pennsylvania state univ, university park, pa)

Local Regularized Least-Square Dimensionality Reduction

A New Reduced-Rank Linear Discriminant Analysis Method and Its Applications

Linear and Nonlinear Dimensionality Reduction from Fluid Mechanics to Machine Learning

Nonlinear Non-Negative Matrix Factorization Using Deep Learning.

Linear Dimensionality Reduction Method Based on Topological Properties

Local Nonlinear Dimensionality Reduction via Preserving the Geometric Structure of Data