The L1/2 regularization method for variable selection in the Cox model

LiuCheng,LiangYong,LuanXin-Ze,LeungKwong-Sak,ChanTak-Ming,XuZong-Ben,ZhangHai

IF: 8.7

2014-01-01

Applied Soft Computing

Abstract:In this paper, we investigate to use the L"1"/"2 regularization method for variable selection based on the Coxu0027s proportional hazards model. The L"1"/"2 regularization can be taken as a representat...

What problem does this paper attempt to address?

The L1/2 regularization method for variable selection in the Cox model.

Cheng Liu,Yong Liang,Xin-Ze Luan,Kwong-Sak Leung,Tak-Ming Chan,Zong-Ben Xu,Hai Zhang

DOI: https://doi.org/10.1016/j.asoc.2013.09.006

IF: 8.7

2014-01-01

Applied Soft Computing

Abstract:In this paper, we investigate to use the L"1"/"2 regularization method for variable selection based on the Cox's proportional hazards model. The L"1"/"2 regularization can be taken as a representative of L"q (0
Iterative L1/2 Regularization Algorithm for Variable Selection in the Cox Proportional Hazards Model

Cheng Liu,Yong Liang,Xin-Ze Luan,Kwong-Sak Leung,Tak-Ming Chan,Zong-Ben Xu,Hai Zhang

DOI: https://doi.org/10.1007/978-3-642-31020-1_2

2012-01-01

Abstract:In this paper, we investigate to use theL 1/2 regularization method for variable selection based on the Cox’s proportional hazards model. The L 1/2 regularization method isa reweighed iterative algorithm with the adaptively weighted L 1 penalty on regression coefficients. The algorithm of theL 1/2 regularization method can be easily obtained by a series of L 1 penalties. Simulation results based on standard artificial data show that theL 1/2 regularization method can be more accurate for variable selection than Lasso and adaptive Lasso methods. The results from Primary Biliary Cirrhosis (PBC) dataset indicate theL 1/2 regularization method performs competitively.
A Novel L1/2 Regularization Shooting Method for Cox’s Proportional Hazards Model

Xin-Ze Luan,Yong Liang,Cheng Liu,Kwong-Sak Leung,Tak-Ming Chan,Zong-Ben Xu,Hai Zhang

DOI: https://doi.org/10.1007/s00500-013-1042-6

IF: 3.732

2013-01-01

Soft Computing

Abstract:Nowadays, a series of methods are based on a L 1 penalty to solve the variable selection problem for a Cox’s proportional hazards model. In 2010, Xu et al. have proposed a L 1/2 regularization and proved that the L 1/2 penalty is sparser than the L 1 penalty in linear regression models. In this paper, we propose a novel shooting method for the L 1/2 regularization and apply it on the Cox model for variable selection. The experimental results based on comprehensive simulation studies, real Primary Biliary Cirrhosis and diffuse large B cell lymphoma datasets show that the L 1/2 regularization shooting method performs competitively.
Adaptive L1/2 shooting regularization method for survival analysis using gene expression data

Xiao-Ying Liu,Yong Liang,Zong-Ben Xu,Hai Zhang,Kwong-Sak Leung

DOI: https://doi.org/10.1155/2013/475702

2013-01-01

Abstract:A new adaptive L-1/2 shooting regularization method for variable selection based on the Cox's proportional hazards mode being proposed. This adaptive L-1/2 shooting algorithm can be easily obtained by the optimization of a reweighed iterative series of L-1 penalties and a shooting strategy of L-1/2 penalty. Simulation results based on high dimensional artificial data show that the adaptive L-1/2 shooting regularization method can be more accurate for variable selection than Lasso and adaptive Lasso methods. The results from real gene expression dataset (DLBCL) also indicate that the L-1/2 regularization method performs competitively.
PENALIZED VARIABLE SELECTION PROCEDURE FOR COX MODELS WITH SEMIPARAMETRIC RELATIVE RISK

Pang Du,Shuangge Ma,Hua Liang

DOI: https://doi.org/10.1214/09-AOS780

2010-08-01

Abstract:We study the Cox models with semiparametric relative risk, which can be partially linear with one nonparametric component, or multiple additive or nonadditive nonparametric components. A penalized partial likelihood procedure is proposed to simultaneously estimate the parameters and select variables for both the parametric and the nonparametric parts. Two penalties are applied sequentially. The first penalty, governing the smoothness of the multivariate nonlinear covariate effect function, provides a smoothing spline ANOVA framework that is exploited to derive an empirical model selection tool for the nonparametric part. The second penalty, either the smoothly-clipped-absolute-deviation (SCAD) penalty or the adaptive LASSO penalty, achieves variable selection in the parametric part. We show that the resulting estimator of the parametric part possesses the oracle property, and that the estimator of the nonparametric part achieves the optimal rate of convergence. The proposed procedures are shown to work well in simulation experiments, and then applied to a real data example on sexually transmitted diseases.
Improving the Robustness of Variable Selection and Predictive Performance of Regularized Generalized Linear Models and Cox Proportional Hazard Models

Feng Hong,Lu Tian,Viswanath Devanarayan

DOI: https://doi.org/10.3390/math11030557

Abstract:High-dimensional data applications often entail the use of various statistical and machine-learning algorithms to identify an optimal signature based on biomarkers and other patient characteristics that predicts the desired clinical outcome in biomedical research. Both the composition and predictive performance of such biomarker signatures are critical in various biomedical research applications. In the presence of a large number of features, however, a conventional regression analysis approach fails to yield a good prediction model. A widely used remedy is to introduce regularization in fitting the relevant regression model. In particular, a L1 penalty on the regression coefficients is extremely useful, and very efficient numerical algorithms have been developed for fitting such models with different types of responses. This L1-based regularization tends to generate a parsimonious prediction model with promising prediction performance, i.e., feature selection is achieved along with construction of the prediction model. The variable selection, and hence the composition of the signature, as well as the prediction performance of the model depend on the choice of the penalty parameter used in the L1 regularization. The penalty parameter is often chosen by K-fold cross-validation. However, such an algorithm tends to be unstable and may yield very different choices of the penalty parameter across multiple runs on the same dataset. In addition, the predictive performance estimates from the internal cross-validation procedure in this algorithm tend to be inflated. In this paper, we propose a Monte Carlo approach to improve the robustness of regularization parameter selection, along with an additional cross-validation wrapper for objectively evaluating the predictive performance of the final model. We demonstrate the improvements via simulations and illustrate the application via a real dataset.
Variable Selection Using the Lasso-Cox Model with Bayesian Regularization

Wenxin Lu,Zhuliang Yu,Zhenghui Gu,Jinhong Huang,Wei Gao,Haiyu Zhou

DOI: https://doi.org/10.1109/iciea.2018.8397844

2018-01-01

Abstract:Selection of prognostic genes associated with tumor has been a subject of considerable research in recent years. In order to solve the high-dimensional gene expression profiles, the Lasso-Cox model has been proposed and widely used in survival analysis. Based on the sparse regression algorithm, the regularization parameter must be carefully tuned by cross-validation to optimize performance. In this paper, we introduce an algorithm based on simple Bayesian approach to replace the process of parameter selection, and the regularization parameter is determined adaptively in training. Simulation results show that variable selection of Bayesian-Lasso (BLasso) can be more accurate than that of Lasso method. We also apply our algorithm to a real dataset DLBCL, and the selected genes have been proven to have close relationship with the tumor.
Regularization and variable selection for infinite variance autoregressive models

Ganggang Xu,Yanbiao Xiang,Suojin Wang,Zhengyan Lin

DOI: https://doi.org/10.1016/j.jspi.2012.03.014

IF: 1.095

2012-01-01

Journal of Statistical Planning and Inference

Abstract:Autoregressive models with infinite variance are of great importance in modeling heavy-tailed time series and have been well studied. In this paper, we propose a penalized method to conduct model selection for autoregressive models with innovations having Pareto-like distributions with index α∈(0,2). By combining the least absolute deviation loss function and the adaptive lasso penalty, the proposed method is able to consistently identify the true model and at the same time produce efficient estimators with a convergence rate of n−1/α. In addition, our approach provides a unified way to conduct variable selection for autoregressive models with finite or infinite variance. A simulation study and a real data analysis are conducted to illustrate the effectiveness of our method.
The variable selection by the Dantzig selector for Cox’s proportional hazards model

Kou Fujimori

DOI: https://doi.org/10.1007/s10463-021-00807-1

2021-08-31

Annals of the Institute of Statistical Mathematics

Abstract:The proportional hazards model proposed by D. R. Cox in a high-dimensional and sparse setting is discussed. The regression parameter is estimated by the Dantzig selector, which will be proved to have the variable selection consistency. This fact enables us to reduce the dimension of the parameter and to construct asymptotically normal estimators for the regression parameter and the cumulative baseline hazard function.

statistics & probability
Variable selection for nonlinear Cox regression model via deep learning

Kexuan Li

DOI: https://doi.org/10.48550/arXiv.2211.09287

2022-11-17

Abstract:Variable selection problem for the nonlinear Cox regression model is considered. In survival analysis, one main objective is to identify the covariates that are associated with the risk of experiencing the event of interest. The Cox proportional hazard model is being used extensively in survival analysis in studying the relationship between survival times and covariates, where the model assumes that the covariate has a log-linear effect on the hazard function. However, this linearity assumption may not be satisfied in practice. In order to extract a representative subset of features, various variable selection approaches have been proposed for survival data under the linear Cox model. However, there exists little literature on variable selection for the nonlinear Cox model. To break this gap, we extend the recently developed deep learning-based variable selection model LassoNet to survival data. Simulations are provided to demonstrate the validity and effectiveness of the proposed method. Finally, we apply the proposed methodology to analyze a real data set on diffuse large B-cell lymphoma.

Machine Learning,Methodology
Estimation and variable selection in a joint model of survival times and longitudinal outcomes with random effects

Antoine Caillebotte,Estelle Kuhn,Sarah Lemler

2024-07-22

Abstract:This paper considers a joint survival and mixed-effects model to explain the survival time from longitudinal data and high-dimensional covariates. The longitudinal data is modeled using a nonlinear effects model, where the regression function serves as a link function incorporated into a Cox model as a covariate. In that way, the longitudinal data is related to the survival time at a given time. Additionally, the Cox model takes into account the inclusion of high-dimensional covariates. The main objectives of this research are two-fold: first, to identify the relevant covariates that contribute to explaining survival time, and second, to estimate all unknown parameters of the joint model. For that purpose, we consider the maximization of a Lasso penalized likelihood. To tackle the optimization problem, we implement a pre-conditioned stochastic gradient to handle the latent variables of the nonlinear mixed-effects model associated with a proximal operator to manage the non-differentiability of the penalty. We provide relevant simulations that showcase the performance of the proposed variable selection and parameters' estimation method in the joint modeling of a Cox and logistic model.

Statistics Theory
High-dimensional variable selection for Cox's proportional hazards model

Jianqing Fan,Yang Feng,Yichao Wu

DOI: https://doi.org/10.48550/arXiv.1002.3315

IF: 5.414

2010-02-17

Machine Learning

Abstract:Variable selection in high dimensional space has challenged many contemporary statistical problems from many frontiers of scientific disciplines. Recent technology advance has made it possible to collect a huge amount of covariate information such as microarray, proteomic and SNP data via bioimaging technology while observing survival information on patients in clinical studies. Thus, the same challenge applies to the survival analysis in order to understand the association between genomics information and clinical information about the survival time. In this work, we extend the sure screening procedure Fan and Lv (2008) to Cox's proportional hazards model with an iterative version available. Numerical simulation studies have shown encouraging performance of the proposed method in comparison with other techniques such as LASSO. This demonstrates the utility and versatility of the iterative sure independent screening scheme.
On the Maximum Penalized Full Likelihood Approach for Cox Model with Extreme Value for Heavily Censored Survival Data

Huazhen Yu,Lixin Zhang

DOI: https://doi.org/10.1016/j.spl.2023.109880

IF: 0.718

2023-01-01

Statistics & Probability Letters

Abstract:In this paper, to deal with heavily censored survival data, a penalized full likelihood (PFL) with extreme value is proposed to estimate the regression coefficients and baseline hazard function in Cox model simultaneously, where multiple penalties are used for variable selection. We present a single-loop algorithm to fit the tail of the baseline distribution beyond a threshold with an extreme value model. The proposed maximum PFL estimators are proved to possess good asymptotic properties, which are validated by simulations and real data analysis.
Cox's proportional hazards model with a high-dimensional and sparse regression parameter

Kou Fujimori

DOI: https://doi.org/10.48550/arXiv.1710.10416

2017-10-28

Abstract:This paper deals with the proportional hazards model proposed by D. R. Cox in a high-dimensional and sparse setting for a regression parameter. To estimate the regression parameter, the Dantzig selector is applied. The variable selection consistency of the Dantzig selector for the model will be proved. This property enables us to reduce the dimension of the parameter and to construct asymptotically normal estimators for the regression parameter and the cumulative baseline hazard function.

Statistics Theory
Variable Selection for Generalized Linear Models with Interval-Censored Failure Time Data

Rong Liu,Shishun Zhao,Tao Hu,Jianguo Sun

DOI: https://doi.org/10.3390/math10050763

IF: 2.4

2022-02-27

Mathematics

Abstract:Variable selection is often needed in many fields and has been discussed by many authors in various situations. This is especially the case under linear models and when one observes complete data. Among others, one common situation where variable selection is required is to identify important risk factors from a large number of covariates. In this paper, we consider the problem when one observes interval-censored failure time data arising from generalized linear models, for which there does not seem to exist an established method. To address this, we propose a penalized least squares method with the use of an unbiased transformation and the oracle property of the method is established along with the asymptotic normality of the resulting estimators of regression parameters. Simulation studies were conducted and demonstrated that the proposed method performed well for practical situations. In addition, the method was applied to a motivating example about children’s mortality data of Nigeria.

mathematics
Bayesian Variable Selection for Cox Regression Model with Spatially Varying Coefficients with Applications to Louisiana Respiratory Cancer Data

Jinjian Mu,Qingyang Liu,Lynn Kuo,Guanyu Hu

DOI: https://doi.org/10.1002/bimj.202000047

2021-07-29

Abstract:The Cox regression model is a commonly used model in survival analysis. In public health studies, clinical data are often collected from medical service providers of different locations. There are large geographical variations in the covariate effects on survival rates from particular diseases. In this paper, we focus on the variable selection issue for the Cox regression model with spatially varying coefficients. We propose a Bayesian hierarchical model which incorporates a horseshoe prior for sparsity and a point mass mixture prior to determine whether a regression coefficient is spatially varying. An efficient two-stage computational method is used for posterior inference and variable selection. It essentially applies the existing method for maximizing the partial likelihood for the Cox model by site independently first, and then applying an MCMC algorithm for variable selection based on results of the first stage. Extensive simulation studies are carried out to examine the empirical performance of the proposed method. Finally, we apply the proposed methodology to analyzing a real data set on respiratory cancer in Louisiana from the SEER program.

Applications
Simultaneous variable selection and estimation for survival data via the Gaussian seamless‐L0 penalty

Zili Liu,Hong Wang

DOI: https://doi.org/10.1002/sim.10031

2024-02-08

Statistics in Medicine

Abstract:We propose a new simultaneous variable selection and estimation procedure with the Gaussian seamless‐L0 (GSELO) penalty for Cox proportional hazard model and additive hazards model. The GSELO procedure shows good potential to improve the existing variable selection methods by taking strength from both best subset selection (BSS) and regularization. In addition, we develop an iterative algorithm to implement the proposed procedure in a computationally efficient way. Theoretically, we establish the convergence properties of the algorithm and asymptotic theoretical properties of the proposed procedure. Since parameter tuning is crucial to the performance of the GSELO procedure, we also propose an extended Bayesian information criteria (EBIC) parameter selector for the GSELO procedure. Simulated and real data studies have demonstrated the prediction performance and effectiveness of the proposed method over several state‐of‐the‐art methods.

public, environmental & occupational health,medicine, research & experimental,medical informatics,mathematical & computational biology,statistics & probability
Variable selection and structure identification for varying coefficient Cox models

Toshio Honda,Ryota Yabe

DOI: https://doi.org/10.48550/arXiv.1607.05415

2016-07-19

Abstract:We consider varying coefficient Cox models with high-dimensional covariates. We apply the group Lasso method to these models and propose a variable selection procedure. Our procedure copes with variable selection and structure identification from a high dimensional varying coefficient model to a semivarying coefficient model simultaneously. We derive an oracle inequality and closely examine restrictive eigenvalue conditions, too. In this paper, we give the details for Cox models with time-varying coefficients. The theoretical results on variable selection can be easily extended to some other important models and we briefly mention those models since those models can be treated in the same way. The models considered in this paper are the most popular models among structured nonparametric regression models. The results of a small numerical study are also given.

Statistics Theory
Robust variable selection methods with Cox model—a selective practical benchmark study

Yunwei Zhang,Samuel Muller

DOI: https://doi.org/10.1093/bib/bbae508

IF: 9.5

2024-10-15

Briefings in Bioinformatics

Abstract:With the advancement of biological and medical techniques, we can now obtain large amounts of high-dimensional omics data with censored survival information. This presents challenges in method development across various domains, particularly in variable selection. Given the inherently skewed distribution of the survival time outcome variable, robust variable selection methods offer potential solutions. Recently, there has been a focus on extending robust variable selection methods from linear regression models to survival models. However, despite these developments, robust methods are currently rarely used in practical applications, possibly due to a limited appreciation of their overall good performance. To address this gap, we conduct a selective review comparing the variable selection performance of twelve robust and non-robust penalised Cox models. Our study reveals the intricate relationship among covariates, survival outcomes, and modeling approaches, demonstrating how subtle variations can significantly impact the performance of methods considered. Based on our empirical research, we recommend the use of robust Cox models for variable selection in practice based on their superior performance in presence of outliers while maintaining good efficiency and accuracy when there are no outliers. This study provides valuable insights for method development and application, contributing to a better understanding of the relationship between correlated covariates and censored outcomes.

biochemical research methods,mathematical & computational biology
Oracle Inequalities for the Lasso in the Cox Model

Jian Huang,Tingni Sun,Zhiliang Ying,Yi Yu,Cun-Hui Zhang

DOI: https://doi.org/10.1214/13-aos1098

2013-01-01

Abstract:We study the absolute penalized maximum partial likelihood estimator in sparse, high-dimensional Cox proportional hazards regression models where the number of time-dependent covariates can be larger than the sample size. We establish oracle inequalities based on natural extensions of the compatibility and cone invertibility factors of the Hessian matrix at the true regression coefficients. Similar results based on an extension of the restricted eigenvalue can be also proved by our method. However, the presented oracle inequalities are sharper since the compatibility and cone invertibility factors are always greater than the corresponding restricted eigenvalue. In the Cox regression model, the Hessian matrix is based on time-dependent covariates in censored risk sets, so that the compatibility and cone invertibility factors, and the restricted eigenvalue as well, are random variables even when they are evaluated for the Hessian at the true regression coefficients. Under mild conditions, we prove that these quantities are bounded from below by positive constants for time-dependent covariates, including cases where the number of covariates is of greater order than the sample size. Consequently, the compatibility and cone invertibility factors can be treated as positive constants in our oracle inequalities.

The L1/2 regularization method for variable selection in the Cox model

The L1/2 regularization method for variable selection in the Cox model.

Iterative L1/2 Regularization Algorithm for Variable Selection in the Cox Proportional Hazards Model

A Novel L1/2 Regularization Shooting Method for Cox’s Proportional Hazards Model

Adaptive L1/2 shooting regularization method for survival analysis using gene expression data

PENALIZED VARIABLE SELECTION PROCEDURE FOR COX MODELS WITH SEMIPARAMETRIC RELATIVE RISK

Improving the Robustness of Variable Selection and Predictive Performance of Regularized Generalized Linear Models and Cox Proportional Hazard Models

Variable Selection Using the Lasso-Cox Model with Bayesian Regularization

Regularization and variable selection for infinite variance autoregressive models

The variable selection by the Dantzig selector for Cox’s proportional hazards model

Variable selection for nonlinear Cox regression model via deep learning

Estimation and variable selection in a joint model of survival times and longitudinal outcomes with random effects

High-dimensional variable selection for Cox's proportional hazards model

On the Maximum Penalized Full Likelihood Approach for Cox Model with Extreme Value for Heavily Censored Survival Data

Cox's proportional hazards model with a high-dimensional and sparse regression parameter

Variable Selection for Generalized Linear Models with Interval-Censored Failure Time Data

Bayesian Variable Selection for Cox Regression Model with Spatially Varying Coefficients with Applications to Louisiana Respiratory Cancer Data

Simultaneous variable selection and estimation for survival data via the Gaussian seamless‐L0 penalty

Variable selection and structure identification for varying coefficient Cox models

Robust variable selection methods with Cox model—a selective practical benchmark study

Oracle Inequalities for the Lasso in the Cox Model