Abstract:Classical Linear Discriminant Analysis (LDA) is not applicable for small sample size problems due to the singularity of the scatter matrices involved. Regularized LDA (RLDA) provides a simple strategy to overcome the singularity problem by applying a regularization term, which is commonly estimated via cross-validation from a set of candidates. However, cross-validation may be computationally prohibitive when the candidate set is large. An efficient algorithm for RLDA is presented that computes the optimal transformation of RLDA for a large set of parameter candidates, with approximately the same cost as running RLDA a small number of times. Thus it facilitates efficient model selection for RLDA.An intrinsic relationship between RLDA and Uncorrelated LDA (ULDA), which was recently proposed for dimension reduction and classification is presented. More specifically, RLDA is shown to approach ULDA when the regularization value tends to zero. That is, RLDA without any regularization is equivalent to ULDA. It can be further shown that ULDA maps all data points from the same class to a common point, under a mild condition which has been shown to hold for many high-dimensional datasets. This leads to the overfitting problem in ULDA, which has been observed in several applications. Thetheoretical analysis presented provides further justification for the use of regularization in RLDA. Extensive experiments confirm the claimed theoretical estimate of efficiency. Experiments also show that, for a properly chosen regularization parameter, RLDA performs favorably in classification, in comparison with ULDA, as well as other existing LDA-based algorithms and Support Vector Machines (SVM).

Optimal Regularization Parameter Estimation for Spectral Regression Discriminant Analysis

Robust spectral regression for face recognition

How to Estimate the Regularization Parameter for Spectral Regression Discriminant Analysis and Its Kernel Version?

Head Pose Estimation Using Spectral Regression Discriminant Analysis

Spectral Regression for Efficient Regularized Subspace Learning

Regularized orthogonal forward feature selection for spectral data

An Adaptive Regularization Method for Sparse Representation

Spectral regression: a regression framework for efficient regularized subspace learning

Sparse Regularization Discriminant Analysis for Face Recognition

SRDA: an Efficient Algorithm for Large-Scale Discriminant Analysis

Joint Spectral Regression Methods for Large-Scale Discriminant Analysis

Efficient Kernel Discriminant Analysis via Spectral Regression

SRDA: an Efficient Algorithm for Large-Scale

An Approach of Regularization Parameter Estimation for Sparse Signal Recovery

Efficient model selection for regularized linear discriminant analysis

The Influence of Regularization Parameter on Error Bound in Super-Resolution Reconstruction

A Criterion for the Construction of a Regularization Function in Sparse Component Analysis

Speed up kernel discriminant analysis

Regularized Coplanar Discriminant Analysis for Dimensionality Reduction

A novel one-parameter regularized linear discriminant analysis for solving small sample size problem in face recognition

Regularized Locality Preserving Discriminant Analysis