Abstract:Partial least squares (PLS) regression has achieved desirable performance for modeling the relationship between a set of dependent (response) variables with another set of independent (predictor) variables, especially when the sample size is small relative to the dimension of these variables. In each iteration, PLS finds two latent variables from a set of dependent and independent variables via maximizing the product of three factors: variances of the two latent variables as well as the square of the correlation between these two latent variables. In this paper, we derived the mathematical formulation of the relationship between mean square error (MSE) and these three factors. We find that MSE is not monotonous with the product of the three factors. However, the corresponding optimization problem is difficult to solve if we extract the optimal latent variables directly based on this relationship. To address these problems, a novel multilinear regression model-variance constrained partial least squares (VCPLS) is proposed. In the proposed VCPLS, we find the latent variables via maximizing the product of the variance of latent variable from dependent variables and the square of the correlation between the two latent variables, while constraining the variance of the latent variable from independent variables must be larger than a predetermined threshold. The corresponding optimization problem can be solved computational efficiently, and the latent variables extracted by VCPLS are near-optimal. Compared with classical PLS and it is variants, VCPLS can achieve lower prediction error in the sense of MSE. The experiments are conducted on three near-infrared spectroscopy (NIR) data sets. To demonstrate the applicability of our proposed VCPLS, we also conducted experiments on another data set, which has different characteristics from NIR data. Experimental results verified the superiority of our proposed VCPLS.

Transformed Partial Least Squares for Multivariate Data

Understanding the Non-Stationary Relationships Between Corn Yields and Meteorology Via a Spatiotemporally Varying Coefficient Model

A penalized likelihood approach for efficiently estimating a partially linear additive transformation model with current status data

Hierarchical Total Variations and Doubly Penalized ANOVA Modeling for Multivariate Nonparametric Regression

An adaptive strategy to improve the partial least squares model via minimum covariance determinant

Sparse Model Identification and Learning for Ultra-high-dimensional Additive Partially Linear Models

Semiparametric Small Area Estimation of Crop Acreage under Partially Linear Model

Penalized Partial Least Squares Based on B-Splines Transformations

Partial Least Squares Regression Trees for Multivariate Response Data With Multicollinear Predictors

Variance constrained partial least squares

New Partially Linear Regression and Machine Learning Models Applied to Agronomic Data

Multilinear and Nonlinear Generalizations of Partial Least Squares: an Overview of Recent Advances

Spatial Heterogeneous Additive Partial Linear Model: A Joint Approach of Bivariate Spline and Forest Lasso

A non-linear partial least squares based on monotonic inner relation

Variable selection for partially linear models via Bayesian subset modeling with diffusing prior

Multivariate Spatiotemporal Models with Low Rank Coefficient Matrix

A dimension reduction based approach for estimation and variable selection in partially linear single-index models with high-dimensional covariates

Penalized Spline Estimation for Partially Linear Single-Index Models

Partial Least Squares Methods: Partial Least Squares Correlation and Partial Least Square Regression

Model selection for varying coefficient nonparametric transformation model

Sparse partial least squares regression for simultaneous dimension reduction and variable selection