Abstract:Dimensionality reduction has many applications in data visualization and machine learning. Existing methods can be classified into global ones and local ones. The global methods usually learn the linear relationship in data, while the local ones learn the manifold intrinsic geometry structure, which has a significant impact on pattern recognition. However, most of existing local methods obtain an embedding with eigenvalue or singular value decomposition, where the computational complexities are very high in a large amount of high-dimensional data. In this paper, we propose a local nonlinear dimensionality reduction method named Vec2vec , which employs a neural network with only one hidden layer to reduce the computational complexity. We first build a neighborhood similarity graph from the input matrix, and then define the context of data points with the random walk properties in the graph. Finally, we train the neural network with the context of data points to learn the embedding of the matrix. We conduct extensive experiments of data classification and clustering on nine image and text datasets to evaluate the performance of our method. Experimental results show that Vec2vec is better than several state-of-the-art dimensionality reduction methods, except that it is equivalent to UMAP on data clustering tasks in the statistical hypothesis tests, but Vec2vec needs less computational time than UMAP in high-dimensional data. Furthermore, we propose a more lightweight method named Approximate Vec2vec (AVec2vec) with little performance degradation, which employs an approximate method to build the neighborhood similarity graph. AVec2vec is still better than some state-of-the-art local dimensionality reduction methods and competitive with UMAP on data classification and clustering tasks in the statistical hypothesis tests.

Improved Locally Linear Embedding by Cognitive Geometry

Locality Pursuit Embedding

Image Feature Optimization Based on Nonlinear Dimensionality Reduction

Learning a Locality Preserving Subspace for Visual Recognition.

Image retrieval using dimensionality reduction

Deep Manifold Computing and Visualization Using Elastic Locally Isometric Smoothness

AN IMPROVED ALGORITHM ON LOCALLY LINEAR EMBEDDING

Small World Neighborhood Optimized Local Linear Embedding Algorithm

MLLE: Modified Locally Linear Embedding Using Multiple Weights

Nonlinear embedding preserving multiple local-linearities

Markov-Lipschitz Deep Learning

An Iterative Locally Linear Embedding Algorithm

Local Nonlinear Dimensionality Reduction via Preserving the Geometric Structure of Data

Robust Hessian Locally Linear Embedding Techniques for High-Dimensional Data

Neighbor line-based locally linear embedding

A Geometry-Aware Algorithm to Learn Hierarchical Embeddings in Hyperbolic Space

Boundary Detection Algorithm Inspired by Locally Linear Embedding

Global structure-guided neighborhood preserving embedding for dimensionality reduction

A Local Similarity-Preserving Framework for Nonlinear Dimensionality Reduction with Neural Networks

Flexible Orthogonal Neighborhood Preserving Embedding.

Hierarchical Neighbors Embedding