Abstract:Linear discriminant analysis (LDA) has been a useful tool in pattern recognition and data analysis research and practice. While linearity of class boundaries cannot always be expected, nonlinear projections through pre-trained deep neural networks have served to map complex data onto feature spaces in which linear discrimination has served well. The solution to binary LDA is obtained by eigenvalue analysis of within-class and between-class scatter matrices. It is well known that the multiclass LDA is solved by an extension to the binary LDA, a generalised eigenvalue problem, from which the largest subspace that can be extracted is of dimension one lower than the number of classes in the given problem. In this paper, we show that, apart from the first of the discriminant directions, the generalised eigenanalysis solution to multiclass LDA does neither yield orthogonal discriminant directions nor maximise discrimination of projected data along them. Surprisingly, to the best of our knowledge, this has not been noted in decades of literature on LDA. To overcome this drawback, we present a derivation with a strict theoretical support for sequentially obtaining discriminant directions that are orthogonal to previously computed ones and maximise in each step the Fisher criterion. We show distributions of projections along these axes and demonstrate that discrimination of data projected onto these discriminant directions has optimal separation, which is much higher than those from the generalised eigenvectors of the multiclass LDA. Using a wide range of benchmark tasks, we present a comprehensive empirical demonstration that on a number of pattern recognition and classification problems, the optimal discriminant subspaces obtained by the proposed method, referred to as GO-LDA (Generalised Optimal LDA), can offer superior accuracy.

Genetic generalized discriminant analysis and its applications

Generalized Discriminant Analysis: A Matrix Exponential Approach

Class-Incremental Generalized Discriminant Analysis

A Modified Algorithm for Generalized Discriminant Analysis

Linear discriminant analysis based on genetic algorithm

Global subclass discriminant analysis

Kernel Generalized Nonlinear Discriminant Analysis Algorithm For Pattern Recognition

An Improved Generalized Discriminant Analysis for Large-Scale Data Set

A Simple and Effective Generalization of Exponential Matrix Discriminant Analysis and Its Application to Face Recognition.

GO-LDA: Generalised Optimal Linear Discriminant Analysis

An Analytical Algorithm for Determining the Generalized Optimal Set of Discriminant Vectors.

GD-RDA: A New Regularized Discriminant Analysis for High-Dimensional Data

Graph regularized linear discriminant analysis and its generalization

Generalized two-dimensional linear discriminant analysis with regularization

A Novel Fisher Discriminant Approach Based On Genetic Algorithm

Generalized Nonlinear Discriminant Analysis and Its Small Sample Size Problems

Generalized discriminant analysis model and its extension for facial expression recognition

A New Algorithm for Generalized Optimal Discriminant Vectors

Fast Algorithms for the Generalized Foley-Sammon Discriminant Analysis

By Normalizing To Improve Generalized Foley-Sammon Transform In High-Dimensional Spaces-----With Application To Face Recognition

Optimal Feature Selection for Sparse Linear Discriminant Analysis and Its Applications in Gene Expression Data