Abstract:Linear Discriminant Analysis (LDA) has been a popular method for extracting features that preserves class separability. The projection functions of LDA are commonly obtained by maximizing the between-class covariance and simultaneously minimizing the within-class covariance. It has been widely used in many fields of information processing, such as machine learning, data mining, information retrieval, and pattern recognition. However, the computation of LDA involves dense matrices eigendecomposition, which can be computationally expensive in both time and memory. Specifically, LDA has Oðmnt þ t3Þ time complexity and requires Oðmn þ mt þ ntÞ memory, where m is the number of samples, n is the number of features, and t ¼ minðm; nÞ. When both m and n are large, it is infeasible to apply LDA. In this paper, we propose a novel algorithm for discriminant analysis, called Spectral Regression Discriminant Analysis (SRDA). By using spectral graph analysis, SRDA casts discriminant analysis into a regression framework that facilitates both efficient computation and the use of regularization techniques. Specifically, SRDA only needs to solve a set of regularized least squares problems, and there is no eigenvector computation involved, which is a huge save of both time and memory. Our theoretical analysis shows that SRDA can be computed with OðmsÞ time and OðmsÞ memory, where sð� nÞ is the average number of nonzero features in each sample. Extensive experimental results on four real-world data sets demonstrate the effectiveness and efficiency of our algorithm.

Statistical Uncorrelation Analysis

Theoretical Analysis of Direct LDA in Small Sample Size Problem

Discriminative analysis of direct linear discriminant analysis for face recognition

Two Dimension Linear Discriminant Analysis Based on Image Sampling and Regroupment

Constrained Linear Discrimination Analysis and Face Recognition

Two-Stage Regularized Linear Discriminant Analysis for 2-D Data.

Data-Dependent Kernel Discriminant Analysis for Feature Extraction and Classification

Orthogonal Vs. Uncorrelated Least Squares Discriminant Analysis for Feature Extraction.

Improved kernel direct Fisher discriminant analysis and face recognition

Making Fisher Discriminant Analysis Scalable.

A NEW ALGORITHM FOR COMPUTING OPTIMAL DISCRIMINANT VECTORS IN CONDITION OF HIGH-DIMENSIONAL DATA

General Averaged Divergence Analysis

A Generalized Form of Fisher Linear Discriminant Function

Linear Discriminant Analysis Using Rotational Invariant L1 Norm

A Flexible and Efficient Algorithm for Regularized Fisher Discriminant Analysis.

Neighborhood linear discriminant analysis

A Variant Fisher Discriminant Analysis Algorithm and Its Applicationto Fault Diagnosis.

SRDA: an Efficient Algorithm for Large-Scale

Linear Discriminant Regularized Regression

SRDA: an Efficient Algorithm for Large-Scale Discriminant Analysis

Covariance Discriminative Learning: A Natural and Efficient Approach to Image Set Classification