Abstract:Recent advances have shown that algorithms with (2D) matrix-based representation perform better than the traditional (1D) vector-based ones. In particular, 2D-LDA has been widely reported to outperform 1D-LDA. However, would the matrix-based linear discriminant analysis be always superior and when would 1D-LDA be better? In this paper, we investigate into these questions and have a comprehensive comparison between 1D-LDA and 2D-LDA in theory and in experiments. We analyze the heteroscedastic problem in 2D-LDA and formulate mathematical equalities to explore the relationship between 1D-LDA and 2D-LDA; then we point out potential problems in 2D-LDA. It is shown that 2D-LDA has eliminated the information contained in the covariance information between different local geometric structures, such as the rows or the columns, which is useful for discriminant feature extraction, whereas 1D-LDA could preserve such information. Interestingly, this new finding indicates that 1D-LDA is able to gain higher Fisher score than 2D-LDA in some extreme case. Furthermore, sufficient conditions on which 2D-LDA would be Bayes optimal for two-class classification problem are derived and comparison with 1D-LDA in this aspect is also analyzed. This could help understand how 2D-LDA is expected to achieve at its best, further discover its relationship with 1D-LDA, and well support other findings. After the theoretical analysis, comprehensive experimental results are reported by fairly and extensively comparing 1D-LDA with 2D-LDA. In contrast to the existing view that some 2D-LDA based algorithms would perform better than 1D-LDA when the number of training samples for each class is small or when the number of discriminant features used is small, we show that it is not always true and show that some standard 1D-LDA based algorithms could perform better in those cases on some challenging data sets.

Matrix-based Vs. Vector-Based Linear Discriminant Analysis: A Comparison of Regularized Variants on Multivariate Time Series Data.

Regularized Bilinear Discriminant Analysis for Multivariate Time Series Data

Pseudo Bidirectional Linear Discriminant Analysis For Multivariate Time Series Classification

Discriminative analysis of direct linear discriminant analysis for face recognition

1D-LDA vs. 2D-LDA: When is vector-based linear discriminant analysis better than matrix-based?

Two-Stage Regularized Linear Discriminant Analysis for 2-D Data.

Regularized Linear Discriminant Analysis Using a Nonlinear Covariance Matrix Estimator

MBLDA: A Novel Multiple Between-Class Linear Discriminant Analysis

1 D-LDA versus 2 D-LDA : When Is Vector-based Linear Discriminant Analysis Better than Matrix-based ?

Linear Discriminant Regularized Regression

Probabilistic Linear Discriminant Analysis with Vectorial Representation for Tensor Data.

Discriminative Matrix-Variate Restricted Boltzmann Machine Classification Model

Training Linear Discriminant Analysis in Linear Time

Regularized Deep Linear Discriminant Analysis

Robust sparse manifold discriminant analysis

Sr-LDA:Sparse and Reduced-Rank Linear Discriminant Analysis for High Dimensional Matrix

Block Linear Discriminant Analysis for Visual Tensor Objects with Frequency or Time Information

A Note on Two-Dimensional Linear Discriminant Analysis.

SRDA: an Efficient Algorithm for Large-Scale Discriminant Analysis

A Novel Linear Discriminant Analysis Based on Alternate Ratio Sum Minimization