Abstract:Linear discriminant analysis (LDA) has been a popular method for dimensionality reduction, which preserves class separability. The projection vectors are commonly obtained by maximizing the between-class covariance and simultaneously minimizing the within-class covariance. LDA can be performed either in the original input space or in the reproducing kernel Hilbert space (RKHS) into which data points are mapped, which leads to kernel discriminant analysis (KDA). When the data are highly nonlinear distributed, KDA can achieve better performance than LDA. However, computing the projective functions in KDA involves eigen-decomposition of kernel matrix, which is very expensive when a large number of training samples exist. In this paper, we present a new algorithm for kernel discriminant analysis, called Spectral Regression Kernel Discriminant Analysis (SRKDA). By using spectral graph analysis, SRKDA casts discriminant analysis into a regression framework, which facilitates both efficient computation and the use of regularization techniques. Specifically, SRKDA only needs to solve a set of regularized regression problems, and there is no eigenvector computation involved, which is a huge save of computational cost. The new formulation makes it very easy to develop incremental version of the algorithm, which can fully utilize the computational results of the existing training samples. Moreover, it is easy to produce sparse projections (Sparse KDA) with a L 1 -norm regularizer. Extensive experiments on spoken letter, handwritten digit image and face image data demonstrate the effectiveness and efficiency of the proposed algorithm.

Joint Spectral Regression Methods for Large-Scale Discriminant Analysis

SRDA: an Efficient Algorithm for Large-Scale Discriminant Analysis

SRDA: an Efficient Algorithm for Large-Scale

Spectral regression: a regression framework for efficient regularized subspace learning

Efficient Kernel Discriminant Analysis via Spectral Regression

Joint Learning Dictionary and Discriminative Features for High Dimensional Data.

Optimal Regularization Parameter Estimation for Spectral Regression Discriminant Analysis

HYPERSPECTRAL IMAGE CLASSIFICATION VIA A JOINT SPARSITY AND SPATIAL CORRELATION MODEL

Spectral Regression: A Unified Approach for Sparse Subspace Learning

Low Rank Spectral Regression Via Matrix Factorization for Efficient Subspace Learning

Speed up kernel discriminant analysis

How to Estimate the Regularization Parameter for Spectral Regression Discriminant Analysis and Its Kernel Version?

Spectral Regression for Efficient Regularized Subspace Learning

Joint Low-Rank Representation and Spectral Regression for Robust Subspace Learning.

Training Linear Discriminant Analysis in Linear Time

Head Pose Estimation Using Spectral Regression Discriminant Analysis

Robust spectral regression for face recognition

Sequential scaled sparse factor regression

Joint Principal Component and Discriminant Analysis for Dimensionality Reduction

Joint Sparse Local Linear Discriminant Analysis for Feature Dimensionality Reduction of Hyperspectral Images

Robust sparse manifold discriminant analysis