Abstract:Variable selection is of crucial significance in QSAR modeling since it increases the model predictive ability and reduces noise. The selection of the right variables is far more complicated than the development of predictive models. In this study, eight continuous and categorical data sets were employed to explore the applicability of two distinct variable selection methods random forests (RF) and least absolute shrinkage and selection operator (LASSO). Variable selection was performed: (1) by using recursive random forests to rule out a quarter of the least important descriptors at each iteration and (2) by using LASSO modeling with 10-fold inner cross-validation to tune its penalty λ for each data set. Along with regular statistical parameters of model performance, we proposed the highest pairwise correlation rate, average pairwise Pearson's correlation coefficient, and Tanimoto coefficient to evaluate the optimal by RF and LASSO in an extensive way. Results showed that variable selection could allow a tremendous reduction of noisy descriptors (at most 96% with RF method in this study) and apparently enhance model's predictive performance as well. Furthermore, random forests showed property of gathering important predictors without restricting their pairwise correlation, which is contrary to LASSO. The mutual exclusion of highly correlated variables in LASSO modeling tends to skip important variables that are highly related to response endpoints and thus undermine the model's predictive performance. The optimal variables selected by RF share low similarity with those by LASSO (e.g., the Tanimoto coefficients were smaller than 0.20 in seven out of eight data sets). We found that the differences between RF and LASSO predictive performances mainly resulted from the variables selected by different strategies rather than the learning algorithms. Our study showed that the right selection of variables is more important than the learning algorithm for modeling. We hope that a standard procedure could be developed based on these proposed statistical metrics to select the truly important variables for model interpretation, as well as for further use to facilitate drug discovery and environmental toxicity assessment.

Variable selection and model prediction based on Lasso, adaptive lasso and elastic net

Adaptive Lasso in high-dimensional settings

Model selection via standard error adjusted adaptive lasso

Recursive Random Forests Enable Better Predictive Performance and Model Interpretation than Variable Selection by LASSO

Variable selection for uncertain regression models based on elastic net method

Repeated Sieving for Prediction Model Building with High-Dimensional Data

On LASSO for predictive regression

Variable Selection in Credit Risk Models for Chinese Listed Companies

Study of Bayesian variable selection method on mixed linear regression models

Robust Variable Selection Based on Relaxed Lad Lasso

High-dimensional regression in practice: an empirical study of finite-sample prediction, variable selection and ranking

Model selection and estimation in regression with grouped variables

On Lasso and adaptive Lasso for non-random sample in credit scoring

Variable Selection in Logistic Regression Model with Genetic Algorithm.

Variable Selection and Minimax Prediction in High-dimensional Functional Linear Model

Post Selection Shrinkage Estimation for High Dimensional Data Analysis

Variable Selection for Linear Mixed Effects Model Via Penalization Approaches

Integrating complex selection rules into the latent overlapping group Lasso for constructing coherent prediction models

Variable selection with missing data in both covariates and outcomes: Imputation and machine learning

Robust Regression Shrinkage and Consistent Variable Selection Through the LAD-Lasso

Elastic Net Procedure for Partially Linear Models