Abstract:High-dimensional data applications often entail the use of various statistical and machine-learning algorithms to identify an optimal signature based on biomarkers and other patient characteristics that predicts the desired clinical outcome in biomedical research. Both the composition and predictive performance of such biomarker signatures are critical in various biomedical research applications. In the presence of a large number of features, however, a conventional regression analysis approach fails to yield a good prediction model. A widely used remedy is to introduce regularization in fitting the relevant regression model. In particular, a L1 penalty on the regression coefficients is extremely useful, and very efficient numerical algorithms have been developed for fitting such models with different types of responses. This L1-based regularization tends to generate a parsimonious prediction model with promising prediction performance, i.e., feature selection is achieved along with construction of the prediction model. The variable selection, and hence the composition of the signature, as well as the prediction performance of the model depend on the choice of the penalty parameter used in the L1 regularization. The penalty parameter is often chosen by K-fold cross-validation. However, such an algorithm tends to be unstable and may yield very different choices of the penalty parameter across multiple runs on the same dataset. In addition, the predictive performance estimates from the internal cross-validation procedure in this algorithm tend to be inflated. In this paper, we propose a Monte Carlo approach to improve the robustness of regularization parameter selection, along with an additional cross-validation wrapper for objectively evaluating the predictive performance of the final model. We demonstrate the improvements via simulations and illustrate the application via a real dataset.

Selecting Penalty Parameters of High-Dimensional M-Estimators using Bootstrapping after Cross-Validation

Variable Selection in Expectile Regression

Wild bootstrap inference for penalized quantile regression for longitudinal data

Cross-validatory Framework for Optimal Parameter Estimation of KPCA and KPLS Models

Penalized Regression Methods With Modified Cross‐Validation and Bootstrap Tuning Produce Better Prediction Models

Tuning parameter selection in econometrics

On Neighbourhood Cross Validation

Bootstrapping the Cross-Validation Estimate

An Alternating Direction Method of Multipliers for MCP-penalized Regression with High-dimensional Data

On the Asymptotic Optimality of Cross-Validation based Hyper-parameter Estimators for Regularized Least Squares Regression Problems

Bootstrapping the Out-of-sample Predictions for Efficient and Accurate Cross-Validation

Improving the Robustness of Variable Selection and Predictive Performance of Regularized Generalized Linear Models and Cox Proportional Hazard Models

Penalty Parameter Selection in Deconvolution by Estimating the Risk for a Smaller Sample Size

Robust Variable Selection Via Nonconcave Penalties with an Upgraded Parsimonious Dynamic Covariance Modeling

Concentration inequalities of the cross-validation estimator for Empirical Risk Minimiser

Proximal Iteration for Nonlinear Adaptive Lasso

Stability-Adjusted Cross-Validation for Sparse Linear Regression

Gradient-based bilevel optimization for multi-penalty Ridge regression through matrix differential calculus

Simultaneous Estimation and Dataset Selection for Transfer Learning in High Dimensions by a Non-convex Penalty

A High-dimensional M-estimator Framework for Bi-level Variable Selection

Penalized robust estimators in logistic regression with applications to sparse models