Abstract:Recursive concave-convex Fisher Linear Discriminant (RPFLD) is a novel efficient dimension reduction method and has been successfully applied to image recognition. However, RPFLD suffers from singularity problem and may lose some useful discriminant information when applied to high-dimensional data. Moreover, RPFLD is computationally expensive because it has to solve a series of quadratic programming (QP) problems to obtain optimal solution. In order to improve the generalization performance of RPFLD and at the same time reduce its training burden, we propose a novel method termed as regularized least squares Fisher linear discriminant (RLS-FLD) in this paper. The central idea is to introduce regularization into RPFLD and simultaneously use the 2-norm loss function. In doing so, the objective function of RLS-FLD turns out to be positive-definite, thus avoiding singularity problem. To solve RLS-FLD, the concave-convex programming (CCP) algorithm is employed to convert the original nonconvex problem to a series of equality-constrained convex QP problems. Each optimization problem in this series has a closed-form solution in its primal formulation via classic Lagrangian method. The resulting RLS-FLD thus leads to much fast training speed and does not need any optimization packages. Meanwhile, theoretical analysis is provided to uncover the connections between RLS-FLD and regularized linear discriminant analysis (RLDA), thus giving more insight into the principle of RLS-FLD. The effectiveness of the proposed RLS-FLD is demonstrated by experimental results on some real-world handwritten digit, face and object recognition datasets.

Relational Fisher Analysis: A General Framework For Dimensionality Reduction

Relational Fisher Analysis: Dimensionality Reduction in Relational Data with Global Convergence

Nonparametric Marginal Fisher Analysis for Feature Extraction

Fisher Difference Discriminant Analysis: Determining the Effective Discriminant Subspace Dimensions for Face Recognition

Spectral Regression Based Marginal Fisher Analysis Dimensionality Reduction Algorithm.

Dimensionality Reduction of Hyperspectral Images with Local Geometric Structure Fisher Analysis

Theory of Fisher Linear Discriminant Analysis and Its Application

Making Fisher Discriminant Analysis Scalable.

Adaptive Quasiconformal Kernel Fisher Discriminant Analysis Via Weighted Maximum Margin Criterion

Feature Selection Based on Sparse Fisher Discrimimant Analysis

lfda: An R Package for Local Fisher Discriminant Analysis and Visualization

Local Fisher Discriminant Analysis with Locally Linear Embedding Affinity Matrix

Dimensionality Reduction by Integrating Sparse Representation and Fisher Criterion and Its Applications

Regularized least squares fisher linear discriminant with applications to image recognition

Local Geometric Structure Feature For Dimensionality Reduction Of Hyperspectral Imagery

Locality Preserving Fisher Discriminant Analysis for Face Recognition

A Linear Discriminant Analysis Framework Based on Random Subspace for Face Recognition.

Hierarchical Multi-view Fisher Discriminant Analysis

A General Framework for Dimensionality Reduction of K-Means Clustering

Marginal Fisher Analysis with Polynomial Matrix Function

Two-Dimensional fisher discriminant analysis and its application to face recognition