Abstract:High-dimensional data applications often entail the use of various statistical and machine-learning algorithms to identify an optimal signature based on biomarkers and other patient characteristics that predicts the desired clinical outcome in biomedical research. Both the composition and predictive performance of such biomarker signatures are critical in various biomedical research applications. In the presence of a large number of features, however, a conventional regression analysis approach fails to yield a good prediction model. A widely used remedy is to introduce regularization in fitting the relevant regression model. In particular, a L1 penalty on the regression coefficients is extremely useful, and very efficient numerical algorithms have been developed for fitting such models with different types of responses. This L1-based regularization tends to generate a parsimonious prediction model with promising prediction performance, i.e., feature selection is achieved along with construction of the prediction model. The variable selection, and hence the composition of the signature, as well as the prediction performance of the model depend on the choice of the penalty parameter used in the L1 regularization. The penalty parameter is often chosen by K-fold cross-validation. However, such an algorithm tends to be unstable and may yield very different choices of the penalty parameter across multiple runs on the same dataset. In addition, the predictive performance estimates from the internal cross-validation procedure in this algorithm tend to be inflated. In this paper, we propose a Monte Carlo approach to improve the robustness of regularization parameter selection, along with an additional cross-validation wrapper for objectively evaluating the predictive performance of the final model. We demonstrate the improvements via simulations and illustrate the application via a real dataset.

Robust multi-outcome regression with correlated covariate blocks using fused LAD-lasso

Reducing bias and mitigating the influence of excess of zeros in regression covariates with multi-outcome adaptive LAD-lasso

LAD-LASSO: SIMULATION STUDY OF ROBUST REGRESSION IN HIGH DIMENSIONAL DATA

Robust adaptive LASSO in high-dimensional logistic regression

Robust Fused Lasso Penalized Huber Regression with Nonasymptotic Property and Implementation Studies

Longitudinal LASSO: Jointly Learning Features and Temporal Contingency for Outcome Prediction

Multi-block linearized alternating direction method for sparse fused Lasso modeling problems

Joint Probability Estimation of Many Binary Outcomes via Localized Adversarial Lasso

Handling correlated and repeated measurements with the smoothed multivariate square-root Lasso

AN EXTENDED LINEARIZED ALTERNATING DIRECTION METHOD OF MULTIPLIERS FOR FUSED-LASSO PENALIZED LINEAR REGRESSION

A Tuning-free Robust and Efficient Approach to High-dimensional Regression

Outlier detection and robust variable selection via the penalized weighted LAD-LASSO method

Efficient Inference on High-Dimensional Linear Models with Missing Outcomes

The out-of-sample prediction error of the square-root-LASSO and related estimators

The joint graphical lasso for inverse covariance estimation across multiple classes

An Accelerated Proximal Alternating Direction Method of Multipliers for Robust Fused Lasso

Variable selection for both outcomes and predictors: sparse multivariate principal covariates regression

Fused LASSO Penalized Least Absolute Deviation Estimator for High Dimensional Linear Regression

Robust Regression Shrinkage and Consistent Variable Selection Through the LAD-Lasso

Robust Variable Selection Based on Relaxed Lad Lasso

Improving the Robustness of Variable Selection and Predictive Performance of Regularized Generalized Linear Models and Cox Proportional Hazard Models