Abstract:High-dimensional data applications often entail the use of various statistical and machine-learning algorithms to identify an optimal signature based on biomarkers and other patient characteristics that predicts the desired clinical outcome in biomedical research. Both the composition and predictive performance of such biomarker signatures are critical in various biomedical research applications. In the presence of a large number of features, however, a conventional regression analysis approach fails to yield a good prediction model. A widely used remedy is to introduce regularization in fitting the relevant regression model. In particular, a L1 penalty on the regression coefficients is extremely useful, and very efficient numerical algorithms have been developed for fitting such models with different types of responses. This L1-based regularization tends to generate a parsimonious prediction model with promising prediction performance, i.e., feature selection is achieved along with construction of the prediction model. The variable selection, and hence the composition of the signature, as well as the prediction performance of the model depend on the choice of the penalty parameter used in the L1 regularization. The penalty parameter is often chosen by K-fold cross-validation. However, such an algorithm tends to be unstable and may yield very different choices of the penalty parameter across multiple runs on the same dataset. In addition, the predictive performance estimates from the internal cross-validation procedure in this algorithm tend to be inflated. In this paper, we propose a Monte Carlo approach to improve the robustness of regularization parameter selection, along with an additional cross-validation wrapper for objectively evaluating the predictive performance of the final model. We demonstrate the improvements via simulations and illustrate the application via a real dataset.

CLASSIFICATION WITH POLYNOMIAL KERNELS AND $L^1-$Coefficient REGULARIZATION

Learning Rates for Regularized Classifiers Using Multivariate Polynomial Kernels

Support Vector Machines Regression with L1-Regularizer

Learning Rates of Least-Square Regularized Regression with Polynomial Kernels

Generalization Analysis of Fredholm Kernel Regularized Classifiers

Concentration Estimates for Learning with ℓ1-Regularizer and Data Dependent Hypothesis Spaces

Learning Theory Estimates for Coefficient-Based Regularized Regression

Learning with coefficient-based regularization and ℓ 1 −penalty

Regularization in kernel learning

Regularized Multinomial Regression Method for Hyperspectral Data Classification Via Pathwise Coordinate Optimization

Classification with Non-I.i.d. Sampling

Robust classification using l2,1-norm based regression model

Quantile Regression With L(1)-Regularization And Gaussian Kernels

Improving the Robustness of Variable Selection and Predictive Performance of Regularized Generalized Linear Models and Cox Proportional Hazard Models

Learning with Convex Loss and Indefinite Kernels

On the Impact of Regularization Variation on Localized Multiple Kernel Learning.

A local stochastic Lipschitz condition with application to Lasso for high dimensional generalized linear models

Kernel Based Simple Regularized Multiple Criteria Linear Programs for Binary Classification

Full length article: Approximation by multivariate Bernstein-Durrmeyer operators and learning rates of least-squares regularized regression with multivariate polynomial kernels

Learning Analysis of Kernel Ridgeless Regression with Asymmetric Kernel Learning

The Optimality of Kernel Classifiers in Sobolev Space