Abstract:Linear Discriminant Analysis (LDA) has been a popular method for extracting features that preserves class separability. The projection functions of LDA are commonly obtained by maximizing the between-class covariance and simultaneously minimizing the within-class covariance. It has been widely used in many fields of information processing, such as machine learning, data mining, information retrieval, and pattern recognition. However, the computation of LDA involves dense matrices eigendecomposition, which can be computationally expensive in both time and memory. Specifically, LDA has O(mnt + t(3) )time complexity and requires O(mn + mt + nt) memory, where m is the number of samples, n is the number of features, and t = min(m,n). When both m and n are large, it is infeasible to apply LDA. In this paper, we propose a novel algorithm for discriminant analysis, called Spectral Regression Discriminant Analysis (SRDA). By using spectral graph analysis, SRDA casts discriminant analysis into a regression framework that facilitates both efficient computation and the use of regularization techniques. Specifically, SRDA only needs to solve a set of regularized least squares problems, and there is no eigenvector computation involved, which is a huge save of both time and memory. Our theoretical analysis shows that SRDA can be computed with O(ms) time and O(ms) memory, where s(<= n) n is the average number of nonzero features in each sample. Extensive experimental results on four real-world data sets demonstrate the effectiveness and efficiency of our algorithm.

Robust Discriminant Regression for Feature Extraction

Robust spectral regression for face recognition

Low-rank Representation Based Discriminative Projection for Robust Feature Extraction

Locality-regularized Linear Regression Discriminant Analysis for Feature Extraction

Linear Laplacian Discrimination for Feature Extraction

Parameterless Reconstructive Discriminant Analysis for Feature Extraction

Robust feature extraction via information theoretic learning

Low-rank 2D Local Discriminant Graph Embedding for Robust Image Feature Extraction

L1-Norm Kernel Discriminant Analysis Via Bayes Error Bound Optimization for Robust Feature Extraction

Data-Dependent Kernel Discriminant Analysis for Feature Extraction and Classification

Robust discriminative non-negative matrix factorization.

Feature Extraction By Locality-Based Linear Discriminant Analysis

Robust double relaxed regression for image classification

Feature Extraction Based on Low-rank Affinity Matrix for Biological Recognition

Discriminant similarity and variance preserving projection for feature extraction

Regularized Scatter Measure for Linear Feature Extraction

Toward Robust Discriminative Projections Learning Against Adversarial Patch Attacks

Robust sparse low-rank embedding for image dimension reduction

Sparse Fuzzy Two-Dimensional Discriminant Local Preserving Projection (SF2DDLPP) for Robust Image Feature Extraction

Integrated Low-Rank-Based Discriminative Feature Learning for Recognition

SRDA: an Efficient Algorithm for Large-Scale Discriminant Analysis