Abstract:High dimensional data reduction techniques are provided by using partial least squares within deep learning. Our framework provides a nonlinear extension of PLS together with a disciplined approach to feature selection and architecture design in deep learning. This leads to a statistical interpretation of deep learning that is tailor made for predictive problems. We can use the tools of PLS, such as scree-plot, bi-plot to provide model diagnostics. Posterior predictive uncertainty is available using MCMC methods at the last layer. Thus we achieve the best of both worlds: scalability and fast predictive rule construction together with uncertainty quantification. Our key construct is to employ deep learning within PLS by predicting the output scores as a deep learner of the input scores. As with PLS our X-scores are constructed using SVD and applied to both regression and classification problems and are fast and scalable. Following Frank and Friedman 1993, we provide a Bayesian shrinkage interpretation of our nonlinear predictor. We introduce a variety of new partial least squares models: PLS-ReLU, PLS-Autoencoder, PLS-Trees and PLS-GP. To illustrate our methodology, we use simulated examples and the analysis of preferences of orange juice and predicting wine quality as a function of input characteristics. We also illustrate Brillinger's estimation procedure to provide the feature selection and data dimension reduction. Finally, we conclude with directions for future research.

Boosting the Partial Least Square Algorithm for Regression Modelling

Boosting The Pls Algorithm For Regressive Modelling

A Boosting-Partial Least Squares Method For Ultraviolet Spectroscopic Analysis Of Water Quality

Analysis of NIR Using Boosting Algorithm

Maximum Likelihood Estimators in a Two Step Model for PLS

A Regression Algorithm Based on AdaBoost

An Improved Generalized Predictive Control in a Robust Dynamic Partial Least Square Framework

Boosting: An Ensemble Learning Tool for Compound Classification and QSAR Modeling

Robust boosting for regression problems

SPLBoost: An Improved Robust Boosting Algorithm Based on Self-Paced Learning

Boosting regression methods based on a geometric conversion approach: Using SVMs base learners

Penalized Partial Least Squares Based on B-Splines Transformations

Automatic piecewise linear regression

Probabilistic Learning of Partial Least Squares Regression Model: Theory and Industrial Applications

Partial least trimmed squares regression

Feature Extraction in Signal Regression: A Boosting Technique for Functional Data Regression

The Evolution of Boosting Algorithms - From Machine Learning to Statistical Modelling

Deep Learning Partial Least Squares

Broad Minimax Probability Learning System and its Application in Regression Modeling

Boosting Support Vector Machines Successfully.

Feature Learning Viewpoint of AdaBoost and a New Algorithm