Abstract:Dimensionality reduction is a crucial step for pattern recognition and data mining tasks to overcome the curse of dimensionality. Principal component analysis (PCA) is a traditional technique for unsupervised dimensionality reduction, which is often employed to seek a projection to best represent the data in a least-squares sense, but if the original data is nonlinear structure, the performance of PCA will quickly drop. An supervised dimensionality reduction algorithm called Linear discriminant analysis (LDA) seeks for an embedding transformation, which can work well with Gaussian distribution data or single-modal data, but for non-Gaussian distribution data or multimodal data, it gives undesired results. What is worse, the dimension of LDA cannot be more than the number of classes. In order to solve these issues, Local shrunk discriminant analysis (LSDA) is proposed in this work to process the non-Gaussian distribution data or multimodal data, which not only incorporate both the linear and nonlinear structures of original data, but also learn the pattern shrinking to make the data more flexible to fit the manifold structure. Further, LSDA has more strong generalization performance, whose objective function will become local LDA and traditional LDA when different extreme parameters are utilized respectively. What is more, a new efficient optimization algorithm is introduced to solve the non-convex objective function with low computational cost. Compared with other related approaches, such as PCA, LDA and local LDA, the proposed method can derive a subspace which is more suitable for non-Gaussian distribution and real data. Promising experimental results on different kinds of data sets demonstrate the effectiveness of the proposed approach.

A Revised Formation of Trace Ratio LDA for Small Sample Size Problem.

Theoretical Analysis of Direct LDA in Small Sample Size Problem

A Novel Formulation of Trace Ratio Linear Discriminant Analysis

Linear discriminant analysis for the small sample size problem: an overview

Two Dimension Linear Discriminant Analysis Based on Image Sampling and Regroupment

Comments and CorrectionsConvergence Analysis on Trace Ratio Linear Discriminant Analysis Algorithms.

Linear discriminant analysis with spectral regularization

A Novel Linear Discriminant Analysis Based on Alternate Ratio Sum Minimization

Beyond Trace Ratio: Weighted Harmonic Mean of Trace Ratios for Multiclass Discriminant Analysis

Semi-Supervised Dimension Reduction Using Trace Ratio Criterion.

A Convergence Analysis on the Iterative Trace Ratio Algorithm and Its Refinements

Ratio Sum Versus Sum Ratio for Linear Discriminant Analysis

Robust Linear Discriminant Analysis Using Ratio Minimization of L1,2-Norms

Sparse Trace Ratio LDA for Supervised Feature Selection

A New Formulation of Linear Discriminant Analysis for Robust Dimensionality Reduction

Ratio sum formula for dimensionality reduction

Transferable Linear Discriminant Analysis.

Training Linear Discriminant Analysis in Linear Time

Recursive Dimension Reduction for Semisupervised Learning

A new LDA-based face recognition system which can solve the small sample size problem

Local Shrunk Discriminant Analysis (LSDA).