Abstract:The salient progress of deep learning is accompanied by nonnegligible deficiencies, such as: 1) interpretability problem; 2) requirement for large data amounts; 3) hard to design and tune parameters; and 4) heavy computation complexity. Despite the remarkable achievements of neural networks-based deep models in many fields, the practical applications of deep learning are still limited by these shortcomings. This article proposes a new concept called the lightweight deep model (LDM). LDM absorbs the useful ideas of deep learning and overcomes their shortcomings to a certain extent. We explore the idea of LDM from the perspective of partial least squares (PLS) by constructing a deep PLS (DPLS) model. The feasibility and merits of DPLS are proved theoretically, after that, DPLS is further generalized to a more common form (GDPLS) by adding a nonlinear mapping layer between two cascaded PLS layers in the model structure. The superiority of DPLS and GDPLS is demonstrated through four practical cases involving two regression problems and two classification tasks, in which our model not only achieves competitive performance compared with existing neural networks-based deep models but also is proven to be a more interpretable and efficient method, and we know exactly how it improves performance, how it gives correct results. Note that our proposed model can only be regarded as an alternative to fully connected neural networks at present and cannot completely replace the mature deep vision or language models.

OPLS and COPLS: Two New PLS Modeling Approaches

A Two-Step PLS Inspired Method for Linear Prediction with Group Effect

Face Representation and Recognition Using Partial Least Squares Regression

Improved Partial Least Squares and Feature Extraction

Two-Dimensional Partial Least Squares and Its Application in Image Recognition

A Two-Dimensional Partial Least Squares with Application to Biological Image Recognition

Two Dimension Nonnegative Partial Least Squares for Face Recognition

Conjugate orthonormalized partial least squares regression and its application in image recognition

Two dimensional nonnegative partial least squares for face recognition

Two Dimensional Non-Negative Sparse Partial Least Squares for Face Recognition

D$$^{2}$$PLS: A Novel Bilinear Method for Facial Feature Fusion

(Dpls)-P-2: A Novel Bilinear Method For Facial Feature Fusion

Multilinear and Nonlinear Generalizations of Partial Least Squares: an Overview of Recent Advances

Sample Label-Based Pls And Feature Extraction

Face Recognition Based on Partial Least Squares and Support Vector Machine

A Novel Face Recognition Method Based on Principal Component Analysis and Kernel Partial Least Squares.

A novel feature fusion method based on partial least squares regression

Face Feature Extraction and Recognition Via Local Binary Pattern and Two-Dimensional Locality Preserving Projection

Face Recognition Based on LDA and Improved Pairwise-Constrained Multiple Metric Learning Method.

Deep PLS: A Lightweight Deep Learning Model for Interpretable and Efficient Data Analytics

Optimal Weighting Distance-Based Similarity for Locally Weighted PLS Modeling