Supervised nonlinear dimensionality reduction for visualization and classification

GengXin,ZhanDe-Chuan,ZhouZhi-Hua

2005-01-01

Abstract:When performing visualization and classification, people often confront the problem of dimensionality reduction. Isomap is one of the most promising nonlinear dimensionality reduction techniques. H...

What problem does this paper attempt to address?

Supervised nonlinear dimensionality reduction for visualization and classification.

Xin Geng,De-Chuan Zhan,Zhi-Hua Zhou

DOI: https://doi.org/10.1109/TSMCB.2005.850151

2005-01-01

Abstract:When performing visualization and classification, people often confront the problem of dimensionality reduction. Isomap is one of the most promising nonlinear dimensionality reduction techniques. However, when Isomap is applied to real-world data, it shows some limitations, such as being sensitive to noise. In this paper, an improved version of Isomap, namely S-Isomap, is proposed. S-Isomap utilizes class information to guide the procedure of nonlinear dimensionality reduction. Such a kind of procedure is called supervised nonlinear dimensionality reduction. In S-Isomap, the neighborhood graph of the input data is constructed according to a certain kind of dissimilarity between data points, which is specially designed to integrate the class information. The dissimilarity has several good properties which help to discover the true neighborhood of the data and, thus, makes S-Isomap a robust technique for both visualization and classification, especially for real-world problems. In the visualization experiments, S-Isomap is compared with Isomap, LLE, and WeightedIso. The results show that S-Isomap performs the best. In the classification experiments, S-Isomap is used as a preprocess of classification and compared with Isomap, WeightedIso, as well as some other well-established classification methods, including the K-nearest neighbor classifier, BP neural network, J4.8 decision tree, and SVM. The results reveal that S-Isomap excels compared to Isomap and WeightedIso in classification, and it is highly competitive with those well-known classification methods.
Image Feature Optimization Based on Nonlinear Dimensionality Reduction

Rong Zhu,Min Yao

DOI: https://doi.org/10.1631/jzus.a0920310

2009-01-01

Journal of Zhejiang University SCIENCE A

Abstract:Image feature optimization is an important means to deal with high-dimensional image data in image semantic understanding and its applications. We formulate image feature optimization as the establishment of a mapping between high- and low-dimensional space via a five-tuple model. Nonlinear dimensionality reduction based on manifold learning provides a feasible way for solving such a problem. We propose a novel globular neighborhood based locally linear embedding (GNLLE) algorithm using neighborhood update and an incremental neighbor search scheme, which not only can handle sparse datasets but also has strong anti-noise capability and good topological stability. Given that the distance measure adopted in nonlinear dimensionality reduction is usually based on pairwise similarity calculation, we also present a globular neighborhood and path clustering based locally linear embedding (GNPCLLE) algorithm based on path-based clustering. Due to its full consideration of correlations between image data, GNPCLLE can eliminate the distortion of the overall topological structure within the dataset on the manifold. Experimental results on two image sets show the effectiveness and efficiency of the proposed algorithms.
Deep Manifold Transformation for Dimension Reduction

Stan Z. Li,Zelin Zang,Lirong Wu

2020-01-01

Abstract:Nonlinear dimensionality reduction (NLDR) plays an important role in feature extraction and visualization of high dimensional data. It transforms patterns of interest, or manifolds, in data to regions in a lower-dimensional latent space. When the manifolds are highly complicated, the NLDR transformation has to provide sufficient nonlinearity for downstream tasks. This paper proposes a novel method, called {\em deep manifold transformation} (DMT), to tackle this problem. As a multi-layer neural network, DMT can support a higher degree of nonlinearity than single-layer methods. Continuation strategy is applied during training to address the local minimum problem common in manifold learning algorithms. Such a learned DMT network can generalize to unseen data whereas traditional manifold learning methods only provide an embedding of the training data. Extensive experiments, comparisons, and ablation studies demonstrate that DMT can deliver results superior to UMAP and t-SNE, and other leading manifold-based NLDR methods.
Computational and Theoretical Analysis of Supervised Dimensionality Reduction

赵武锋,沈海斌,严晓浪

DOI: https://doi.org/10.3785/j.issn.1008-9497.2009.06.015

2009-01-01

Abstract:In the case of small sample size,classical linear discriminant analysis fails due to the singularity of scatter matrices for supervised dimensionality reduction.Many extensions were proposed in the past to overcome this problem,which in general can be classified into three categories: methods based on the null space of the within-class scatter matrix,ones based on the range space of the total scatter matrix and ones based on other subspace.In order to better understand the characteristics of the algorithms of the former two classes,a computational and theoretical analysis was carried out,and concluded that: under a mild condition which holds in many applications involving high-dimensional data,methods based on the null space of the within-class scatter matrix are equivalent to those based on the range space of the total scatter matrix except two differences of constraints for discriminant vectors and implementation procedure.The comparative results on the face database,ORL and YALE,also confirmed the aforementioned conclusion.
Deep Manifold Computing and Visualization

Stan Z. Li,Zelin Zang,Lirong Wu

2020-01-01

Abstract: The ability to preserve local geometry of highly nonlinear manifolds in high dimensional spaces and properly unfold them into lower dimensional hyperplanes is the key to the success of manifold computing, nonlinear dimensionality reduction (NLDR) and visualization. This paper proposes a novel method, called elastic locally isometric smoothness (ELIS), to empower deep neural networks with such an ability. ELIS requires that a desired metric between points should be preserved across layers in order to preserve local geometry; such a smoothness constraint effectively regularizes vector-based transformations to become well-behaved local metric-preserving homeomorphisms. Moreover, ELIS requires that the smoothness should be imposed in a way to render sufficient flexibility for tackling complicated nonlinearity and non-Euclideanity; this is achieved layer-wisely via nonlinearity in both the similarity and activation functions. The ELIS method incorporates a class of suitable nonlinear similarity functions into a two-way divergence loss and uses hyperparameter continuation in finding optimal solutions. Extensive experiments, comparisons, and ablation study demonstrate that ELIS can deliver results not only superior to UMAP and t-SNE for and visualization but also better than other leading counterparts of manifold and autoencoder learning for NLDR and manifold data generation.
Deep Clustering and Visualization for End-to-End High-Dimensional Data Analysis.

Lirong Wu,Lifan Yuan,Guojiang Zhao,Haitao Lin,Stan Z. Li

DOI: https://doi.org/10.1109/tnnls.2022.3151498

IF: 14.255

2023-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:High-dimensional data analysis for exploration and discovery includes two fundamental tasks: deep clustering and data visualization. When these two associated tasks are done separately, as is often the case thus far, disagreements can occur among the tasks in terms of geometry preservation. Namely, the clustering process is often accompanied by the corruption of the geometric structure, whereas visualization aims to preserve the data geometry for better interpretation. Therefore, how to achieve deep clustering and data visualization in an end-to-end unified framework is an important but challenging problem. In this article, we propose a novel neural network-based method, called deep clustering and visualization (DCV), to accomplish the two associated tasks end-to-end to resolve their disagreements. The DCV framework consists of two nonlinear dimensionality reduction (NLDR) transformations: 1) one from the input data space to latent feature space for clustering and 2) the other from the latent feature space to the final 2-D space for visualization. Importantly, the first NLDR transformation is mainly optimized by one Clustering Loss, allowing arbitrary corruption of the geometric structure for better clustering, while the second NLDR transformation is optimized by one Geometry-Preserving Loss to recover the corrupted geometry for better visualization. Extensive comparative results show that the DCV framework outperforms other leading clustering-visualization algorithms in terms of both quantitative evaluation metrics and qualitative visualization.
Deep Manifold Computing and Visualization Using Elastic Locally Isometric Smoothness

Stan Z. Li,Zelin Zang,Lirong Wu

2021-01-01

Abstract:The ability to preserve local geometry of highly nonlinear manifolds in high dimensional spaces and properly unfold them into lower dimensional hyperplanes is the key to the success of manifold computing, nonlinear dimensionality reduction (NLDR) and visualization. This paper proposes a novel method, called elastic locally isometric smoothness (ELIS), to empower deep neural networks with such an ability. ELIS requires that a desired metric between points should be preserved across layers in order to preserve local geometry; such a smoothness constraint effectively regularizes vector-based transformations to become well-behaved local metric-preserving homeomorphisms. Moreover, ELIS requires that the smoothness should be imposed in a way to render sufficient flexibility for tackling complicated nonlinearity and non-Euclideanity; this is achieved layer-wisely via nonlinearity in both the similarity and activation functions. The ELIS method incorporates a class of suitable nonlinear similarity functions into a two-way divergence loss and uses hyperparameter continuation in finding optimal solutions. Extensive experiments, comparisons, and ablation study demonstrate that ELIS can deliver results not only superior to UMAP and t-SNE for and visualization but also better than other leading counterparts of manifold and autoencoder learning for NLDR and manifold data generation.
A Perception-Driven Approach to Supervised Dimensionality Reduction for Visualization

Yunhai Wang,Kang Feng,Xiaowei Chu,Jian Zhang,Chi-Wing Fu,Michael Sedlmair,Xiaohui Yu,Baoquan Chen

DOI: https://doi.org/10.1109/tvcg.2017.2701829

IF: 5.2

2017-01-01

IEEE Transactions on Visualization and Computer Graphics

Abstract:Dimensionality reduction (DR) is a common strategy for visual analysis of labeled high-dimensional data. Low-dimensional representations of the data help, for instance, to explore the class separability and the spatial distribution of the data. Widely-used unsupervised DR methods like PCA do not aim to maximize the class separation, while supervised DR methods like LDA often assume certain spatial distributions and do not take perceptual capabilities of humans into account. These issues make them ineffective for complicated class structures. Towards filling this gap, we present a perception-driven linear dimensionality reduction approach that maximizes the perceived class separation in projections. Our approach builds on recent developments in perception-based separation measures that have achieved good results in imitating human perception. We extend these measures to be density-aware and incorporate them into a customized simulated annealing algorithm, which can rapidly generate a near optimal DR projection. We demonstrate the effectiveness of our approach by comparing it to state-of-the-art DR methods on 93 datasets, using both quantitative measure and human judgments. We also provide case studies with class-imbalanced and unlabeled data.
Nonlinear Dimensionality Reduction in the Analysis of High Dimensional Medical Data

翁时锋,张长水,张学工

DOI: https://doi.org/10.3321/j.issn:1000-0054.2004.04.014

2004-01-01

Abstract:It was difficult to find the intrinsic structure of high dimensional medical data by traditional technologies. The new method named Isomap was applied to two classic medical datasets, lung cancer gene expression data and breast cancer pathological data. The intrinsic dimensionalities of these two datasets were found to be less than three, so they could be visualized in low dimensional space. Comparison of a standard linear dimensionality reduction method, PCA, with Isomap showed that Isomap gave better performance when calculating the within class distance. Therefore, nonlinear dimensionality reduction technology has potential in the analysis of high dimensional medical data.
Supervised dimensionality reduction for big data

Joshua T. Vogelstein,Eric W. Bridgeford,Minh Tang,Da Zheng,Christopher Douville,Randal Burns,Mauro Maggioni

DOI: https://doi.org/10.1038/s41467-021-23102-2

IF: 16.6

2021-05-17

Nature Communications

Abstract:Abstract To solve key biomedical problems, experimentalists now routinely measure millions or billions of features (dimensions) per sample, with the hope that data science techniques will be able to build accurate data-driven inferences. Because sample sizes are typically orders of magnitude smaller than the dimensionality of these data, valid inferences require finding a low-dimensional representation that preserves the discriminating information (e.g., whether the individual suffers from a particular disease). There is a lack of interpretable supervised dimensionality reduction methods that scale to millions of dimensions with strong statistical theoretical guarantees. We introduce an approach to extending principal components analysis by incorporating class-conditional moment estimates into the low-dimensional projection. The simplest version, Linear Optimal Low-rank projection, incorporates the class-conditional means. We prove, and substantiate with both synthetic and real data benchmarks, that Linear Optimal Low-Rank Projection and its generalizations lead to improved data representations for subsequent classification, while maintaining computational efficiency and scalability. Using multiple brain imaging datasets consisting of more than 150 million features, and several genomics datasets with more than 500,000 features, Linear Optimal Low-Rank Projection outperforms other scalable linear dimensionality reduction techniques in terms of accuracy, while only requiring a few minutes on a standard desktop computer.

multidisciplinary sciences
Nonlinear Supervised Dimensionality Reduction via Smooth Regular Embeddings

Cem Ornek,Elif Vural

DOI: https://doi.org/10.48550/arXiv.1710.07120

2018-05-28

Abstract:The recovery of the intrinsic geometric structures of data collections is an important problem in data analysis. Supervised extensions of several manifold learning approaches have been proposed in the recent years. Meanwhile, existing methods primarily focus on the embedding of the training data, and the generalization of the embedding to initially unseen test data is rather ignored. In this work, we build on recent theoretical results on the generalization performance of supervised manifold learning algorithms. Motivated by these performance bounds, we propose a supervised manifold learning method that computes a nonlinear embedding while constructing a smooth and regular interpolation function that extends the embedding to the whole data space in order to achieve satisfactory generalization. The embedding and the interpolator are jointly learnt such that the Lipschitz regularity of the interpolator is imposed while ensuring the separation between different classes. Experimental results on several image data sets show that the proposed method outperforms traditional classifiers and the supervised dimensionality reduction algorithms in comparison in terms of classification accuracy in most settings.

Computer Vision and Pattern Recognition
Nonlinear Dimensionality Reduction for Data Visualization: An Unsupervised Fuzzy Rule-based Approach

Suchismita Das,Nikhil R. Pal

DOI: https://doi.org/10.1109/TFUZZ.2021.3076583

2020-04-08

Abstract:Here, we propose an unsupervised fuzzy rule-based dimensionality reduction method primarily for data visualization. It considers the following important issues relevant to dimensionality reduction-based data visualization: (i) preservation of neighborhood relationships, (ii) handling data on a non-linear manifold, (iii) the capability of predicting projections for new test data points, (iv) interpretability of the system, and (v) the ability to reject test points if required. For this, we use a first-order Takagi-Sugeno type model. We generate rule antecedents using clusters in the input data. In this context, we also propose a new variant of the Geodesic c-means clustering algorithm. We estimate the rule parameters by minimizing an error function that preserves the inter-point geodesic distances (distances over the manifold) as Euclidean distances on the projected space. We apply the proposed method on three synthetic and three real-world data sets and visually compare the results with four other standard data visualization methods. The obtained results show that the proposed method behaves desirably and performs better than or comparable to the methods compared with. The proposed method is found to be robust to the initial conditions. The predictability of the proposed method for test points is validated by experiments. We also assess the ability of our method to reject output points when it should. Then, we extend this concept to provide a general framework for learning an unsupervised fuzzy model for data projection with different objective functions. To the best of our knowledge, this is the first attempt to manifold learning using unsupervised fuzzy modeling.

Machine Learning,Human-Computer Interaction
Linear Dimensionality Reduction: Survey, Insights, and Generalizations

John P. Cunningham,Zoubin Ghahramani

DOI: https://doi.org/10.48550/arXiv.1406.0873

2016-03-19

Abstract:Linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data, due to their simple geometric interpretations and typically attractive computational properties. These methods capture many data features of interest, such as covariance, dynamical structure, correlation between data sets, input-output relationships, and margin between data classes. Methods have been developed with a variety of names and motivations in many fields, and perhaps as a result the connections between all these methods have not been highlighted. Here we survey methods from this disparate literature as optimization programs over matrix manifolds. We discuss principal component analysis, factor analysis, linear multidimensional scaling, Fisher's linear discriminant analysis, canonical correlations analysis, maximum autocorrelation factors, slow feature analysis, sufficient dimensionality reduction, undercomplete independent component analysis, linear regression, distance metric learning, and more. This optimization framework gives insight to some rarely discussed shortcomings of well-known methods, such as the suboptimality of certain eigenvector solutions. Modern techniques for optimization over matrix manifolds enable a generic linear dimensionality reduction solver, which accepts as input data and an objective to be optimized, and returns, as output, an optimal low-dimensional projection of the data. This simple optimization framework further allows straightforward generalizations and novel variants of classical methods, which we demonstrate here by creating an orthogonal-projection canonical correlations analysis. More broadly, this survey and generic solver suggest that linear dimensionality reduction can move toward becoming a blackbox, objective-agnostic numerical technology.

Machine Learning
Real-valued Multivariate Dimension Reduction: A Survey

Hong-Ming SHAN,Jun-Ping ZHANG

DOI: https://doi.org/10.16383/j.aas.2018.c160812

2018-01-01

Abstract:As one of the classical problems in machine learning,dimension reduction is used for dealing with the curse of dimensionality,speeding up computational efficiency of the algorithm,and improving interpretability as well as visualizing high-dimensional data.Traditional dimension reduction algorithms such as principal component analysis (PCA) and linear discriminant analysis are mainly suitable for unlabeled data or classification data.When the response variables are univariate or multivariate continuous real-valued ones,however,such dimension reduction methods cannot guarantee the effective predictive performance of the reduced subspace.In the recent two decades,researchers have been devoted to studying this issue with different viewpoints,attaining many promising and systemic achievements.Under this background,we will survey the developments of real-valued multivariate dimension reduction in detail.We will also introduce its basic concepts,algorithms and theories,and discuss some potential research directions deserving investigating.
Invertible Manifold Learning for Dimension Reduction

Siyuan Li,Haitao Lin,Zelin Zang,Lirong Wu,Jun Xia,Stan Z. Li

DOI: https://doi.org/10.1007/978-3-030-86523-8_43

2021-01-01

Abstract:Dimension reduction (DR) aims to learn low-dimensional representations of high-dimensional data with the preservation of essential information. In the context of manifold learning, we define that the representation after information-lossless DR preserves the topological and geometric properties of data manifolds formally, and propose a novel two-stage DR method, called invertible manifold learning (inv-ML) to bridge the gap between theoretical information-lossless and practical DR. The first stage includes a homeomorphic sparse coordinate transformation to learn low-dimensional representations without destroying topology and a local isometry constraint to preserve local geometry. In the second stage, a linear compression is implemented for the trade-off between the target dimension and the incurred information loss in excessive DR scenarios. Experiments are conducted on seven datasets with a neural network implementation of inv-ML, called i-ML-Enc. Empirically, i-ML-Enc achieves invertible DR in comparison with typical existing methods as well as reveals the characteristics of the learned manifolds. Through latent space interpolation on real-world datasets, we find that the reliability of tangent space approximated by the local neighborhood is the key to the success of manifold-based DR algorithms.
Learning the Supervised NLDR Mapping for Classification

Zhonglin Lin,Shifeng Weng,Changshui Zhang,Naijiang Lu,Zhimin Xia

DOI: https://doi.org/10.1007/978-3-540-28647-9_148

2004-01-01

Abstract:In researches and experiments, we often work with large volumes of high-dimensional data and regularly confront the problem of dimensionality reduction. Some Non-Linear Dimensionality Reduction (NLDR) methods have been developed for unsupervised datasets. As for supervised datasets, there is a newly developed method called SI-somap, which is capable of discovering structures that underlie complex natural observations. However, SIsomap is limited from the fact that it doesn't provide an explicit mapping from original space to embedded space, and thus can't be applied to classification. To solve this problem, we apply neural network to learning that mapping and then to classify. We test our method on a real world dataset. To prove the effectiveness of our method, we also compare it with a related classification method, Extended Isomap. Experiments show that our proposed method has satisfactory performance.
Deep Dimension Reduction for Supervised Representation Learning

Jian Huang,Yuling Jiao,Xu Liao,Jin Liu,Zhou Yu

DOI: https://doi.org/10.1109/tit.2023.3340658

IF: 2.5

2024-01-01

IEEE Transactions on Information Theory

Abstract:The goal of supervised representation learning is to construct effective data representations for prediction. Among all the characteristics of an ideal nonparametric representation of high-dimensional complex data, sufficiency, low dimensionality and disentanglement are some of the most essential ones. We propose a deep dimension reduction approach to learning representations with these characteristics. The proposed approach is a nonparametric generalization of the sufficient dimension reduction method. We formulate the ideal representation learning task as that of finding a nonparametric representation that minimizes an objective function characterizing conditional independence and promoting disentanglement at the population level. We then estimate the target representation at the sample level nonparametrically using deep neural networks. We show that the estimated deep nonparametric representation is consistent in the sense that its excess risk converges to zero. Our extensive numerical experiments using simulated and real benchmark data demonstrate that the proposed methods have better performance than several existing dimension reduction methods and the standard deep learning models in the context of classification and regression.

computer science, information systems,engineering, electrical & electronic
Supervised Discriminative Sparse PCA with Adaptive Neighbors for Dimensionality Reduction

Zhenhua Shi,Dongrui Wu,Jian Huang,Yu-Kai Wang,Chin-Teng Lin

DOI: https://doi.org/10.48550/arXiv.2001.03103

2020-01-12

Abstract:Dimensionality reduction is an important operation in information visualization, feature extraction, clustering, regression, and classification, especially for processing noisy high dimensional data. However, most existing approaches preserve either the global or the local structure of the data, but not both. Approaches that preserve only the global data structure, such as principal component analysis (PCA), are usually sensitive to outliers. Approaches that preserve only the local data structure, such as locality preserving projections, are usually unsupervised (and hence cannot use label information) and uses a fixed similarity graph. We propose a novel linear dimensionality reduction approach, supervised discriminative sparse PCA with adaptive neighbors (SDSPCAAN), to integrate neighborhood-free supervised discriminative sparse PCA and projected clustering with adaptive neighbors. As a result, both global and local data structures, as well as the label information, are used for better dimensionality reduction. Classification experiments on nine high-dimensional datasets validated the effectiveness and robustness of our proposed SDSPCAAN.

Machine Learning
Linear Low-Rank Approximation and Nonlinear Dimensionality Reduction

Zhenyue Zhang,Hongyuan Zha

DOI: https://doi.org/10.1360/04ys0016

2004-01-01

Science in China Series A Mathematics

Abstract:We present our recent work on both linear and nonlinear data reduction methods and algorithms: for the linear case we discuss results on structure analysis of SVD of columnpartitioned matrices and sparse low-rank approximation; for the nonlinear case we investigate methods for nonlinear dimensionality reduction and manifold learning. The problems we address have attracted great deal of interest in data mining and machine learning.
Revisiting Dimensionality Reduction Techniques for Visual Cluster Analysis: An Empirical Study

Jiazhi Xia,Yuchen Zhang,Jie Song,Yang Chen,Yunhai Wang,Shixia Liu

DOI: https://doi.org/10.1109/tvcg.2021.3114694

IF: 5.2

2022-01-01

IEEE Transactions on Visualization and Computer Graphics

Abstract:Dimensionality Reduction (DR) techniques can generate 2D projections and enable visual exploration of cluster structures of high-dimensional datasets. However, different DR techniques would yield various patterns, which significantly affect the performance of visual cluster analysis tasks. We present the results of a user study that investigates the influence of different DR techniques on visual cluster analysis. Our study focuses on the most concerned property types, namely the linearity and locality, and evaluates twelve representative DR techniques that cover the concerned properties. Four controlled experiments were conducted to evaluate how the DR techniques facilitate the tasks of 1) cluster identification, 2) membership identification, 3) distance comparison, and 4) density comparison, respectively. We also evaluated users' subjective preference of the DR techniques regarding the quality of projected clusters. The results show that: 1) Non-linear and Local techniques are preferred in cluster identification and membership identification; 2) Linear techniques perform better than non-linear techniques in density comparison; 3) UMAP (Uniform Manifold Approximation and Projection) and t-SNE (t-Distributed Stochastic Neighbor Embedding) perform the best in cluster identification and membership identification; 4) NMF (Nonnegative Matrix Factorization) has competitive performance in distance comparison; 5) t-SNLE (t-Distributed Stochastic Neighbor Linear Embedding) has competitive performance in density comparison.

computer science, software engineering

Supervised nonlinear dimensionality reduction for visualization and classification

Supervised nonlinear dimensionality reduction for visualization and classification.

Image Feature Optimization Based on Nonlinear Dimensionality Reduction

Deep Manifold Transformation for Dimension Reduction

Computational and Theoretical Analysis of Supervised Dimensionality Reduction

Deep Manifold Computing and Visualization

Deep Clustering and Visualization for End-to-End High-Dimensional Data Analysis.

Deep Manifold Computing and Visualization Using Elastic Locally Isometric Smoothness

A Perception-Driven Approach to Supervised Dimensionality Reduction for Visualization

Nonlinear Dimensionality Reduction in the Analysis of High Dimensional Medical Data

Supervised dimensionality reduction for big data

Nonlinear Supervised Dimensionality Reduction via Smooth Regular Embeddings

Nonlinear Dimensionality Reduction for Data Visualization: An Unsupervised Fuzzy Rule-based Approach

Linear Dimensionality Reduction: Survey, Insights, and Generalizations

Real-valued Multivariate Dimension Reduction: A Survey

Invertible Manifold Learning for Dimension Reduction

Learning the Supervised NLDR Mapping for Classification

Deep Dimension Reduction for Supervised Representation Learning

Supervised Discriminative Sparse PCA with Adaptive Neighbors for Dimensionality Reduction

Linear Low-Rank Approximation and Nonlinear Dimensionality Reduction

Revisiting Dimensionality Reduction Techniques for Visual Cluster Analysis: An Empirical Study