Abstract:Linear discriminant analysis (LDA) has been a popular method for extracting features which preserve class separability. The projection vectors are commonly obtained by maximizing the between class covariance and simultaneously minimizing the within class covariance. LDA can be performed either in the original input space or in the reproducing kernel Hilbert space (RKHS) into which data points are mapped, which leads to Kernel Discriminant Analysis (KDA). When the data are highly nonlinear distributed, KDA can achieve better performance than LDA. However, computing the projective functions in KDA involves eigen-decomposition of kernel matrix, which is very expensive when a large number of training samples exist. In this paper, we present a new algorithm for kernel discriminant analysis, called spectral regression kernel discriminant analysis (SRKDA). By using spectral graph analysis, SRKDA casts discriminant analysis into a regression framework which facilitates both efficient computation and the use of regularization techniques. Specifically, SRKDA only needs to solve a set of regularized regression problems and there is no eigenvector computation involved, which is a huge save of computational cost. Our computational analysis shows that SRKDA is 27 times faster than the ordinary KDA. Moreover, the new formulation makes it very easy to develop incremental version of the algorithm which can fully utilize the computational results of the existing training samples. Experiments on face recognition demonstrate the effectiveness and efficiency of the proposed algorithm.

Efficient Kernel Selection Via Spectral Analysis.

A Kernel Function Optimization And Selection Algorithm Based On Cost Function Maximization

Randomized Kernel Selection with Spectra of Multilevel Circulant Matrices

Kernel Selection with Spectral Perturbation Stability of Kernel Matrix

A Spectral Feature Selection Approach with Kernelized Fuzzy Rough Sets

Eigenvalues Ratio for Kernel Selection of Kernel Methods

Feature Selection in a Kernel Space

Eigenvalues Perturbation of Integral Operator for Kernel Selection.

TAKES: a fast method to select features in the kernel space.

Bi-criterion Optimization for Kernel Parameters Selection

Approximate Kernel Selection via Matrix Approximation

EFFICIENT KERNEL-BASED VARIABLE SELECTION WITH SPARSISTENCY

Efficient Kernel Discriminant Analysis via Spectral Regression

Combinatorial Kernel Matrix Model Selection Using Feature Distances

Maximum Margin Based Semi-supervised Spectral Kernel Learning.

Automated Spectral Kernel Learning

Kernel Mean Estimation via Spectral Filtering

KERNEL ENTROPY-BASED UNSUPERVISED SPECTRAL FEATURE SELECTION

An Efficient Kernel Evaluation Criterion for Multiclass Classification

Bilinear Analysis for Kernel Selection and Nonlinear Feature Extraction

Spectral Regularized Kernel Two-Sample Tests