Abstract:Linear discriminant analysis (LDA) is among the most popular supervised dimensionality reduction algorithms, which has been largely followed in the fields of pattern recognition and data mining. However, LDA has three major drawbacks. One is the challenge brought by small-sample-size (SSS) problem; second makes it sensitive to outliers due to the use of squared L2-norms in the scatter loss evaluation; the third is the case that the feature loadings in projection matrix are relatively redundant and there is a risk of overfitting. In this paper, we put forward a novel functional expression for LDA, which combines maximum margin criterion (MMC) with a weighted strategy formulated by L1,2-norms to against outliers. Meanwhile, we simultaneously realize the adaptive calculation of weighted intra-class and global centroid to further reduce the influence of outliers, and employ the L2,1-norm to constrain row sparsity so that subspace learning and feature selection could be performed cooperatively. Besides, an effective alternating iterative algorithm is derived and its convergence is verified. From the complexity analysis, our proposed algorithm can deal with large-scale data processing. Our proposed model can address the sensitivity problem of outliers and extract the most representative features while preventing overfitting effectively. Experiments performed on several benchmark databases demonstrate that the proposed algorithm is more effective than some other state-of-the-art methods and has better generalization performance.

Recursive Dimension Reduction Algorithm Based on Margin Discriminant Analysis

Computational and Theoretical Analysis of Supervised Dimensionality Reduction

Recursive support vector machines for dimensionality reduction.

An Optimal Set of Uncorrelated Margin Discriminant Vectors

A Maximum Class Distance Support Vector Machine-Based Algorithm for Recursive Dimension Reduction

Self-centralized Jointly Sparse Maximum Margin Criterion for Robust Dimensionality Reduction.

Complete large margin linear discriminant analysis using mathematical programming approach

Marginal discriminant projections: An adaptable margin discriminant approach to feature reduction and extraction

Reduced Support Vector Machine Based on Margin Vectors

Largest Center-Specific Margin for Dimension Reduction

Scaling cut criterion-based discriminant analysis for supervised dimension reduction

An Adaptive Class Pairwise Dimensionality Reduction Algorithm

Supervised Dimensionality Reduction Methods via Recursive Regression

A Dimension Reduction Algorithm Based on Divergence Balance

Spectral Regression Based Marginal Fisher Analysis Dimensionality Reduction Algorithm.

Large Margin Dimensionality Reduction for Time Series

Discriminative Dimension Reduction Via Maximin Separation Probability Analysis

Max-K-Min Distance Analysis for Dimension Reduction

Max-Min Distance Analysis by Using Sequential SDP Relaxation for Dimension Reduction

Margin Maximum Embedding Discriminant (Mmed) For Feature Extraction And Classification

Margin Distribution Analysis