Abstract:Linear discriminant analysis (LDA) represents a simple yet powerful technique for partitioning a p-dimensional feature vector into one of K classes based on a linear projection learned from N labeled observations. However, it is well-established that in the high-dimensional setting (p > N) the underlying projection estimator degenerates. Moreover, any linear discriminate function involving a large number of features may be difficult to interpret. To ameliorate these issues, two general categories of sparse LDA modifications have been proposed, both to reduce the number of active features and to stabilize the resulting projections. The first, based on optimal scoring, is more straightforward to implement and analyze but has been heavily criticized for its ambiguous connection with the original LDA formulation. In contrast, a second strategy applies sparse penalty functions directly to the original LDA objective but requires additional heuristic trade-off parameters, has unknown global and local minima properties, and requires a greedy sequential optimization procedure. In all cases the choice of sparse regularizer can be important, but no rigorous guidelines have been provided regarding which penalty might be preferable. Against this backdrop, we winnow down the broad space of candidate sparse LDA algorithms and promote a specific selection based on optimal scoring coupled with a particular, complementary sparse regularizer. This overall process ultimately progresses our understanding of sparse LDA in general, while leading to targeted modifications of existing algorithms that produce superior results in practice on three high-dimensional gene data sets.

Sparse L1-norm Two Dimensional Linear Discriminant Analysis Via the Generalized Elastic Net Regularization.

Sparse L1-norm-based linear discriminant analysis

Generalized two-dimensional linear discriminant analysis with regularization

L₂,₁-Norm Regularized Robust and Sparse Linear Discriminant Analysis Via an Alternating Direction Method of Multipliers

Robust L1-norm two-dimensional linear discriminant analysis.

Robust Bilateral Lp-norm Two-Dimensional Linear Discriminant Analysis

L2,1-Norm Regularized Robust and Sparse Linear Discriminant Analysis via an Alternating Direction Method of Multipliers

L1-norm Based Two-dimensional Linear Discriminant Analysis

Efficient and Robust Sparse Linear Discriminant Analysis for Data Classification

Robust Linear Discriminant Analysis Using Ratio Minimization of L1,2-Norms

Face Recognition by Sparse Discriminant Analysis Via Joint L2,1-norm Minimization

Regularized Deep Linear Discriminant Analysis

Robust Two-Dimensional Capped L2, 1-Norm Linear Discriminant Analysis with Regularization and Its Applications on Image Recognition.

Generalized robust linear discriminant analysis for jointly sparse learning

Towards Robust Discriminative Projections Learning via Non-Greedy <i>l</i><sub>2,1</sub>-Norm MinMax

Towards Robust and Sparse Linear Discriminant Analysis for Image Classification

Robust and Efficient Linear Discriminant Analysis With <italic>L</italic><sub>2,1</sub>-Norm for Feature Selection

Robust and Efficient Linear Discriminant Analysis With L2, 1-Norm for Feature Selection

Understanding and Evaluating Sparse Linear Discriminant Analysis

Optimal Feature Selection for Sparse Linear Discriminant Analysis and Its Applications in Gene Expression Data

${L_{2,1}}$ -Norm Discriminant Manifold Learning