Abstract:High-dimensional data applications often entail the use of various statistical and machine-learning algorithms to identify an optimal signature based on biomarkers and other patient characteristics that predicts the desired clinical outcome in biomedical research. Both the composition and predictive performance of such biomarker signatures are critical in various biomedical research applications. In the presence of a large number of features, however, a conventional regression analysis approach fails to yield a good prediction model. A widely used remedy is to introduce regularization in fitting the relevant regression model. In particular, a L1 penalty on the regression coefficients is extremely useful, and very efficient numerical algorithms have been developed for fitting such models with different types of responses. This L1-based regularization tends to generate a parsimonious prediction model with promising prediction performance, i.e., feature selection is achieved along with construction of the prediction model. The variable selection, and hence the composition of the signature, as well as the prediction performance of the model depend on the choice of the penalty parameter used in the L1 regularization. The penalty parameter is often chosen by K-fold cross-validation. However, such an algorithm tends to be unstable and may yield very different choices of the penalty parameter across multiple runs on the same dataset. In addition, the predictive performance estimates from the internal cross-validation procedure in this algorithm tend to be inflated. In this paper, we propose a Monte Carlo approach to improve the robustness of regularization parameter selection, along with an additional cross-validation wrapper for objectively evaluating the predictive performance of the final model. We demonstrate the improvements via simulations and illustrate the application via a real dataset.

Regularized receiver operating characteristic-based logistic regression for grouped variable selection with composite criterion

Grouped Variable Selection Using Area under the ROC with Imbalanced Data

A Descriptive Study of Variable Discretization and Cost-Sensitive Logistic Regression on Imbalanced Credit Data

PENALIZED VARIABLE SELECTION PROCEDURE FOR COX MODELS WITH SEMIPARAMETRIC RELATIVE RISK

Group variable selection via convex Log-Exp-Sum penalty with application to a breast cancer survivor study

Adaptive Stochastic Gradient Boosting Tree with Composite Criterion

Adaptive hypergraph regularized logistic regression model for bioinformatic selection and classification

Improving the Robustness of Variable Selection and Predictive Performance of Regularized Generalized Linear Models and Cox Proportional Hazard Models

Study of Bayesian variable selection method on mixed linear regression models

Application of Bayesian variable selection in logistic regression model

Model selection and estimation in regression with grouped variables

Simultaneous Dimension Reduction and Variable Selection for Multinomial Logistic Regression

Combining regularization and logistic regression model to validate the Q‐matrix for cognitive diagnosis model

A High-dimensional M-estimator Framework for Bi-level Variable Selection

Adaptive ensemble of classifiers with regularization for imbalanced data classification

Composite quantile regression and variable selection of the partial linear single-index models

Logistic regression analysis with standardized markers

Bayesian Sparse Global-Local Shrinkage Regression for Selection of Grouped Variables

ROBUST SUBGROUP IDENTIFICATION

Effect Size Estimation and Misclassification Rate Based Variable Selection in Linear Discriminant Analysis

A one-covariate-at-a-time multiple testing approach to variable selection in additive models