Abstract:Subspace selection approaches are powerful tools in pattern classification and data visualization. One of the most important subspace approaches is the linear dimensionality reduction step in the Fisher's linear discriminant analysis (FLDA), which has been successfully employed in many fields such as biometrics, bioinformatics, and multimedia information management. However, the linear dimensionality reduction step in FLDA has a critical drawback: for a classification task with c classes, if the dimension of the projected subspace is strictly lower than c - 1, the projection to a subspace tends to merge those classes, which are close together in the original feature space. If separate classes are sampled from Gaussian distributions, all with identical covariance matrices, then the linear dimensionality reduction step in FLDA maximizes the mean value of the Kullback-Leibler (KL) divergences between different classes. Based on this viewpoint, the geometric mean for subspace selection is studied in this paper. Three criteria are analyzed: 1) maximization of the geometric mean of the KL divergences, 2) maximization of the geometric mean of the normalized KL divergences, and 3) the combination of 1 and 2. Preliminary experimental results based on synthetic data, UCI Machine Learning Repository, and handwriting digits show that the third criterion is a potential discriminative subspace selection method, which significantly reduces the class separation problem in comparing with the linear dimensionality reduction step in FLDA and its several representative extensions.

Regular simplex criterion: A novel feature extraction criterion

Locality Pursuit Embedding

Two Dimension Linear Discriminant Analysis Based on Image Sampling and Regroupment

Two Dimensional Maximum Margin Criterion

Feature extraction via prototype margin distance maximizing criterion for subspace learning

Local Feature Discriminant Projection

A Variance Minimization Criterion to Feature Selection Using Laplacian Regularization

Efficient Feature Extraction For Image Classification

Locality-regularized Linear Regression Discriminant Analysis for Feature Extraction

Regularized Scatter Measure for Linear Feature Extraction

Face Feature Extraction and Recognition Via Local Binary Pattern and Two-Dimensional Locality Preserving Projection

Face Recognition by Stepwise Nonparametric Margin Maximum Criterion

Discriminant similarity and variance preserving projection for feature extraction

Discriminative feature extraction based on sparse and low-rank representation

Face Recognition Using Data Driven Local Appearance Features

Nonparametric maximum margin criterion for face recognition

Linear Laplacian Discrimination for Feature Extraction

Geometric mean for subspace selection.

Sparse Local Discriminant Projections for Feature Extraction

A Joint Learning Framework for Optimal Feature Extraction and Multi-class SVM

Feature Extraction By Locality-Based Linear Discriminant Analysis