Abstract:Linear discriminant analysis (LDA) represents a simple yet powerful technique for partitioning a p-dimensional feature vector into one of K classes based on a linear projection learned from N labeled observations. However, it is well-established that in the high-dimensional setting (p > N) the underlying projection estimator degenerates. Moreover, any linear discriminate function involving a large number of features may be difficult to interpret. To ameliorate these issues, two general categories of sparse LDA modifications have been proposed, both to reduce the number of active features and to stabilize the resulting projections. The first, based on optimal scoring, is more straightforward to implement and analyze but has been heavily criticized for its ambiguous connection with the original LDA formulation. In contrast, a second strategy applies sparse penalty functions directly to the original LDA objective but requires additional heuristic trade-off parameters, has unknown global and local minima properties, and requires a greedy sequential optimization procedure. In all cases the choice of sparse regularizer can be important, but no rigorous guidelines have been provided regarding which penalty might be preferable. Against this backdrop, we winnow down the broad space of candidate sparse LDA algorithms and promote a specific selection based on optimal scoring coupled with a particular, complementary sparse regularizer. This overall process ultimately progresses our understanding of sparse LDA in general, while leading to targeted modifications of existing algorithms that produce superior results in practice on three high-dimensional gene data sets.

An Efficient Convex Formulation for Reduced-Rank Linear Discriminant Analysis in High Dimensions

A New Reduced-Rank Linear Discriminant Analysis Method and Its Applications

Sr-LDA:Sparse and Reduced-Rank Linear Discriminant Analysis for High Dimensional Matrix

Dimension Reduction of Hyperspectral Images with Sparse Linear Discriminant Analysis

On The Equivalent Of Low-Rank Regressions And Linear Discriminant Analysis Based Regressions

A New Formulation of Linear Discriminant Analysis for Robust Dimensionality Reduction

Optimal Discriminant Analysis in High-Dimensional Latent Factor Models

A Novel Linear Discriminant Analysis Based on Alternate Ratio Sum Minimization

A Direct Estimation Approach to Sparse Linear Discriminant Analysis

A Least Squares Formulation of Multi-Label Linear Discriminant Analysis

Robust reduced-rank regression

Towards the Optimal Discriminant Subspace

Generalized robust linear discriminant analysis for jointly sparse learning

Unified Formulation of Linear Discriminant Analysis Methods and Optimal Parameter Selection

A procedure of linear discrimination analysis with detected sparsity structure for high-dimensional multi-class classification

Simultaneous Dimension Reduction and Variable Selection for Multinomial Logistic Regression

Optimal Feature Selection in High-Dimensional Discriminant Analysis

Understanding and Evaluating Sparse Linear Discriminant Analysis

Ratio Sum Versus Sum Ratio for Linear Discriminant Analysis

High-Dimensional Regularized Discriminant Analysis

The Dantzig Discriminant Analysis with High Dimensional Data