Abstract:Dimensionality reduction plays an important role in computer vision problems since it reduces computational cost and is often capable of yielding more discriminative data representation. In this context, Partial Least Squares (PLS) has presented notable results in tasks such as image classification and neural network optimization. However, PLS is infeasible on large datasets, such as ImageNet, because it requires all the data to be in memory in advance, which is often impractical due to hardware limitations. Additionally, this requirement prevents us from employing PLS on streaming applications where the data are being continuously generated. Motivated by this, we propose a novel incremental PLS, named Covariance-free Incremental Partial Least Squares (CIPLS), which learns a low-dimensional representation of the data using a single sample at a time. In contrast to other state-of-the-art approaches, instead of adopting a partially-discriminative or SGD-based model, we extend Nonlinear Iterative Partial Least Squares (NIPALS) -- the standard algorithm used to compute PLS -- for incremental processing. Among the advantages of this approach are the preservation of discriminative information across all components, the possibility of employing its score matrices for feature selection, and its computational efficiency. We validate CIPLS on face verification and image classification tasks, where it outperforms several other incremental dimensionality reduction techniques. In the context of feature selection, CIPLS achieves comparable results when compared to state-of-the-art techniques.

Incremental Regularized Least Squares for Dimensionality Reduction of Large-Scale Data.

Least Square Incremental Linear Discriminant Analysis

Covariance-free Partial Least Squares: An Incremental Dimensionality Reduction Method

Local Regularized Least-Square Dimensionality Reduction

Least squares online linear discriminant analysis

SRDA: an Efficient Algorithm for Large-Scale Discriminant Analysis

Regularized Deep Linear Discriminant Analysis

Supervised Dimensionality Reduction Methods via Recursive Regression

On low-rank regularized least squares for scalable nonlinear classification

A scalable supervised algorithm for dimensionality reduction on streaming data

3E-LDA

SRDA: an Efficient Algorithm for Large-Scale

Adaptive Local Linear Discriminant Analysis

Sparsity Induced Locality Preserving Projection Approaches for Dimensionality Reduction

Linear discriminant analysis for the small sample size problem: an overview

Supervised Multivariate Learning with Simultaneous Feature Auto-grouping and Dimension Reduction

Supervised dimensionality reduction for big data

Inducible Regularization for Low-Rank Matrix Factorizations for Collaborative Filtering

A Flexible and Efficient Algorithm for Regularized Fisher Discriminant Analysis.

Training Linear Discriminant Analysis in Linear Time

Regularized Locality Preserving Indexing Via Spectral Regression