Abstract:Partial least squares (PLS) is a dimensionality reduction technique introduced in the field of chemometrics and successfully employed in many other areas. The PLS components are obtained by maximizing the covariance between linear combinations of the regressors and of the target variables. In this work, we focus on its application to scalar regression problems. PLS regression consists in finding the least squares predictor that is a linear combination of a subset of the PLS components. Alternatively, PLS regression can be formulated as a least squares problem restricted to a Krylov subspace. This equivalent formulation is employed to analyze the distance between ${\hat{\boldsymbol\beta}\;}_{\mathrm{PLS}}^{\scriptscriptstyle {(L)}}$, the PLS estimator of the vector of coefficients of the linear regression model based on $L$ PLS components, and $\hat{\boldsymbol \beta}_{\mathrm{OLS}}$, the one obtained by ordinary least squares (OLS), as a function of $L$. Specifically, ${\hat{\boldsymbol\beta}\;}_{\mathrm{PLS}}^{\scriptscriptstyle {(L)}}$ is the vector of coefficients in the aforementioned Krylov subspace that is closest to $\hat{\boldsymbol \beta}_{\mathrm{OLS}}$ in terms of the Mahalanobis distance with respect to the covariance matrix of the OLS estimate. We provide a bound on this distance that depends only on the distribution of the eigenvalues of the regressor covariance matrix. Numerical examples on synthetic and real-world data are used to illustrate how the distance between ${\hat{\boldsymbol\beta}\;}_{\mathrm{PLS}}^{\scriptscriptstyle {(L)}}$ and $\hat{\boldsymbol \beta}_{\mathrm{OLS}}$ depends on the number of clusters in which the eigenvalues of the regressor covariance matrix are grouped.

A non-asymptotic upper bound in prediction for the PLS estimator

A non-asymptotic analysis of the single component PLS regression

Maximum Likelihood Estimators in a Two Step Model for PLS

A Two-Step PLS Inspired Method for Linear Prediction with Group Effect

Subsampling for Partial Least-Squares Regression Via an Influence Function

Functional Extreme-PLS

On some limitations of probabilistic models for dimension-reduction: Illustration in the case of probabilistic formulations of partial least squares

Rank-Constrained Least-Squares: Prediction and Inference

A PRESS statistic for two-block partial least squares regression

Prediction Risk and Estimation Risk of the Ridgeless Least Squares Estimator under General Assumptions on Regression Errors

On estimation of multivariate prediction regions in partial least squares regression

Partial least trimmed squares regression

Global Sparse Partial Least Squares

Statistical Inference on Asymptotic Properties of Two Estimators for the Partially Linear Single-Index Models

Relation between PLS and OLS regression in terms of the eigenvalue distribution of the regressor covariance matrix

The Surprising Robustness of Partial Least Squares

Estimation of high-dimensional low-rank matrices

Author Response for "partial Least Median of Squares Regression"

Functional Partial Least-Squares: Optimal Rates and Adaptation

[HDDA] sparse subspace constrained partial least squares

The Elephant in the Room: Predictive Performance of PLS Models