Abstract:Dimensionality reduction is a key preprocessing step for many applications. Until our knowledge, unsupervised approaches such as PCA and ICA do not take label information of the original data into account, so a supervised approach such as Linear discriminant analysis (LDA) performs better on many classification tasks. Unfortunately, the classical LDA approach has shortcomings, such as the well-known small size problem, the heteroscedastic problem and the (C-1) low rank problem. The (C-1) low rank problem greatly limits the dimension of the extracted features. In addition, the calculation of the between-class and within-class scatter matrices in the classical LDA approach actually only takes account of the Mahalanobis distance like covariance distance of data centers and each data class, so if the dataset has very few classes or the data distribution of each class is not Gaussian-like but has some spatial structure in the feature space instead, classical LDA does not work well. In this paper we propose a dimensionality reduction approach which avoids the limitations of classical LDA and improves handling of the between-class scatter matrix. Our approach approach takes the distribution of data in each class into consideration to calculate the projection matrix. It does not assume that the data distribution of each class approximates Gaussian; each can have its own spatial structure. Experiments show that our method can obtain better projection directions than the classical LDA approach and greatly improve the classification accuracy. In addition, our approach is able to reconstruct the original signal well, while the classical LDA approach ignores the reconstruction property.

Enhancing Information Discriminant Analysis: Feature Extraction with Linear Statistical Model and Information-Theoretic Criteria.

Supervised Linear Dimensionality Reduction Based on Factor Analysis and Information-Theoertic Criteria

Information Discriminant Feature Extraction Based on Mutual Information Gradient Optimal Computation

Communications Inspired Linear Discriminant Analysis

Feature extraction by dynamically revising the discriminant information in the objective space

A Method of Discriminative Information Preservation and In-Dimension Distance Minimization Method for Feature Selection

Robust and Efficient Linear Discriminant Analysis With L2, 1-Norm for Feature Selection

Joint Dynamic Manifold and Discriminant Information Learning for Feature Extraction

An Improved Linear Discriminant Analysis with L1-Norm for Robust Feature Extraction

Adaptive Local Linear Discriminant Analysis

Covariance-enhanced Discriminant Analysis.

Robust and Efficient Linear Discriminant Analysis With <italic>L</italic><sub>2,1</sub>-Norm for Feature Selection

An Incremental Dimensionality Reduction Method on Discriminant Information for Pattern Classification.

Optimal Dimensionality Discriminant Analysis and Its Application to Image Recognition

Reconstructable generalized maximum scatter difference discriminant analysis

Adaptive and Fuzzy Locality Discriminant Analysis for Dimensionality Reduction

Discriminant-enhanced neighborhood preserving embedding for dimensionality reduction

An Improve Linear Discriminant Analysis Method Based on Regularization

A novel statistical linear discriminant analysis for image matrix: Two-dimensional fisherfaces

Robust Adaptive Linear Discriminant Analysis With Bidirectional Reconstruction Constraint

Theoretical Analysis and Comparison of Several Criteria on Linear Model Dimension Reduction