Variable selection in high-dimensional varying-coefficient models with global optimality

XueLan,QuAnnie

DOI: https://doi.org/10.5555/2188385.2343707

2012-01-01

Journal of Machine Learning Research

Abstract:The varying-coefficient model is flexible and powerful for modeling the dynamic changes of regression coefficients. It is important to identify significant covariates associated with response varia...

What problem does this paper attempt to address?

Variable Selection for Generalized Varying Coefficient Partially Linear Models with Diverging Number of Parameters

Zheng-yan Lin,Yu-ze Yuan

DOI: https://doi.org/10.1007/s10255-011-0071-0

2012-01-01

Abstract:Semiparametric models with diverging number of predictors arise in many contemporary scientific areas. Variable selection for these models consists of two components: model selection for non-parametric components and selection of significant variables for the parametric portion. In this paper, we consider a variable selection procedure by combining basis function approximation with SCAD penalty. The proposed procedure simultaneously selects significant variables in the parametric components and the nonparametric components. With appropriate selection of tuning parameters, we establish the consistency and sparseness of this procedure.
Variable selection in partially time-varying coefficient models

Degui Li,Jia Chen,Zhengyan Lin

DOI: https://doi.org/10.1080/10485250902912694

2009-01-01

Journal of Nonparametric Statistics

Abstract:A partially time-varying coefficient model is introduced to characterise the nonlinearity and trending phenomenon. To enhance predictability and to select significant variables in the parametric component of the model, the penalised least squares method with the help of the profile local linear technique is developed in this article. The convergence rate and the oracle property of the resulting estimator are established under mild conditions. A remarkable achievement of our results is that it does not require undersmoothing of the nonparametric component. Meanwhile, some extensions of the proposed model and method are also discussed. Furthermore, some numerical examples are provided to show that our theory and method work well in practice.
A Unified Variable Selection Approach for Varying Coefficient Models

Yanlin Tang,Huixia Judy Wang,Zhongyi Zhu,Xinyuan Song

DOI: https://doi.org/10.5705/ss.2010.121

IF: 1.4

2011-01-01

Statistica Sinica

Abstract:In varying coefficient models, three types of variable selection problems are of practical interests: separation of varying and constant effects, selection of variables with nonzero varying effects, and selection of variables with nonzero constant effects. Existing variable selection methods in the literature often focus on only one of the three types. In this paper, we develop a unified variable selection approach for both least squares regression and quantile regression models with possibly varying coefficients. The developed method is carried out by using a two-step iterative procedure based on basis expansion and a double adaptive-LASSO-type penalty. Under some regularity conditions, we show that the proposed procedure is consistent in both variable selection and the separation of varying and constant coefficients. In addition, the estimated varying coefficients possess the optimal convergence rate under the same smoothness assumption, and the estimated constant coefficients have the same asymptotic distribution as their counterparts obtained when the true model is known. Finally, we investigate the finite sample performance of the proposed method through a simulation study and the analysis of the Childhood Malnutrition Data in India.
Variable Selection for High Dimensional Partially Linear Varying Coefficient Errors-in-variables Models

Zhaoliang Wang,Liugen Xue

DOI: https://doi.org/10.15672/hjms.2018.603

2018-01-01

Hacettepe Journal of Mathematics and Statistics

Abstract:In this paper, we consider variable selection procedure for the high dimensional partially linear varying coefficient models where the parametric part covariates are measured with additive errors. The penalized bias-corrected proffile least squares estimators are conducted, and their asymptotic properties are also studied under some regularity conditions. The rate of convergence and the asymptotic normality of the resulting estimates are established. We further demonstrate that, with proper choices of the penalty functions and the regularization parameter, the resulting estimates perform asymptotically as well as an oracle property. Choice of smoothing parameters is also discussed. Finite sample performance of the proposed variable selection procedures is assessed by Monte Carlo simulation studies.
Variable Selection in High-Dimensional Quantile Varying Coefficient Models

Yanlin Tang,Xinyuan Song,Huixia Judy Wang,Zhongyi Zhu

DOI: https://doi.org/10.1016/j.jmva.2013.07.015

IF: 1.387

2013-01-01

Journal of Multivariate Analysis

Abstract:In this paper, we propose a two-stage variable selection procedure for high dimensional quantile varying coefficient models. The proposed method is based on basis function approximation and LASSO-type penalties. We show that the first stage penalized estimator with LASSO penalty reduces the model from ultra-high dimensional to a model that has size close to the true model, but contains the true model as a valid sub model. By applying adaptive LASSO penalty to the reduced model, the second stage excludes the remained irrelevant covariates, leading to an estimator consistent in variable selection. A simulation study and the analysis of a real data demonstrate that the proposed method performs quite well in finite samples, with regard to dimension reduction and variable selection.
Automatic Variable Selection for Varying Coefficient Models with Longitudinal Data

Ruiqin Tian,Liugen Xue,Dengke Xu

DOI: https://doi.org/10.1016/j.spl.2016.07.012

IF: 0.718

2016-01-01

Statistics & Probability Letters

Abstract:We propose a novel variable selection for varying coefficient models with longitudinal data. The theoretical properties of the resulting estimators are established. In addition, simulation studies and a real data set are conducted to evaluate the proposed method.
A New Variable Selection Approach for Varying Coefficient Models

Xue-Jun Ma,Jing-Xiao Zhang

DOI: https://doi.org/10.1007/s00184-015-0543-y

IF: 0.96

2015-01-01

Metrika

Abstract:The varying coefficient models are very important tools to explore the hidden structure between the response variable and its predictors. However, variable selection and identification of varying coefficients of the models are poorly understood. In this paper, we develop a novel method to overcome these difficulties using local polynomial smoothing and the SCAD penalty. Under some regularity conditions, we show that the proposed procedure is consistent in separating the varying coefficients from the constant ones. The resulting estimator can be as efficient as the oracle. Simulation results confirm our theories. Finally, we study the Boston housing data using the proposed method.
Structural identification and variable selection in high-dimensional varying-coefficient models

Yuping Chen,Yang Bai,Wing-Kam Fung

DOI: https://doi.org/10.1080/10485252.2017.1303057

2017-01-01

Journal of Nonparametric Statistics

Abstract:Varying-coefficient models have been widely used to investigate the possible time-dependent effects of covariates when the response variable comes from normal distribution. Much progress has been made for inference and variable selection in the framework of such models. However, the identification of model structure, that is how to identify which covariates have time-varying effects and which have fixed effects, remains a challenging and unsolved problem especially when the dimension of covariates is much larger than the sample size. In this article, we consider the structural identification and variable selection problems in varying-coefficient models for high-dimensional data. Using a modified basis expansion approach and group variable selection methods, we propose a unified procedure to simultaneously identify the model structure, select important variables and estimate the coefficient curves. The unique feature of the proposed approach is that we do not have to specify the model structure in advance, therefore, it is more realistic and appropriate for real data analysis. Asymptotic properties of the proposed estimators have been derived under regular conditions. Furthermore, we evaluate the finite sample performance of the proposed methods with Monte Carlo simulation studies and a real data analysis.
Variable selection for partially linear single-index varying-coefficient model

Lijuan Han,Liugen Xue,Junshan Xie

2024-12-18

Abstract:This paper focuses on variable selection for a partially linear single-index varying-coefficient model. A regularized variable selection procedure by combining basis function approximations with SCAD penalty is proposed. It can simultaneously select significant variables in the parametric and nonparametric components and estimate the nonzero regression coefficients and coefficient functions. The consistency of the variable selection procedure and the oracle property of the penalized least-squares estimators for high-dimensional data are established. Some simulations and the real data analysis are constructed to illustrate the finite sample performances of the proposed method.

Statistics Theory
Variable Selection in High-dimensional Partially Linear Models

Yang Yiping,Xue Liugen

DOI: https://doi.org/10.3969/j.issn.1001-4268.2011.02.007

2009-01-01

Abstract:This paper considers the problem of variable selection in high-dimensional partially linear models.By combining spline method and Dantzig selector or Lasso,the authors simultaneously select variables and estimate parameters.The simulation results show that the proposed methods are better than the existing method when the dimension of parameters is much larger than the number of observation.
Robust exponential squared loss-based variable selection for high-dimensional single-index varying-coefficient model

Yunquan Song,Ling Jian,Lu Lin

DOI: https://doi.org/10.1016/j.cam.2016.05.030

IF: 2.872

2016-12-01

Journal of Computational and Applied Mathematics

Abstract:Robust variable selection procedure through penalized regression has been gaining increased attention in the literature. They can be used to perform variable selection and are expected to yield robust estimates. In this article, we propose a robust variable selection procedure for high-dimensional single-index varying-coefficient model using penalized exponential squared loss. The proposed procedure simultaneously selects significant covariates with functional coefficients and local significant variables with parametric coefficients. With proper choices of penalty functions and regularization parameters, we show the asymptotic normality of the resulting estimate and further demonstrate that the proposed procedures perform as well as an oracle procedure. Our simulation studies reveal that our proposed method performs similarly to the oracle method in terms of the model error and the positive selection rate even in the presence of influential points. In contrast, other existing procedures have a much lower noncausal selection rate. Our analysis unravels the discrepancies of using our robust method versus the other penalized regression method, underscoring the importance of developing and applying robust penalized regression methods.

mathematics, applied
Variable Selection for Fixed Effects Varying Coefficient Models

Gao Rong Li,Heng Lian,Peng Lai,Heng Peng

DOI: https://doi.org/10.1007/s10114-015-3159-2

2014-01-01

Acta Mathematica Sinica English Series

Abstract:We consider the problem of variable selection for the fixed effects varying coefficient models. A variable selection procedure is developed using basis function approximations and group nonconcave penalized functions, and the fixed effects are removed using the proper weight matrices. The proposed procedure simultaneously removes the fixed individual effects, selects the significant variables and estimates the nonzero coefficient functions. With appropriate selection of the tuning parameters, an asymptotic theory for the resulting estimates is established under suitable conditions. Simulation studies are carried out to assess the performance of our proposed method, and a real data set is analyzed for further illustration.
Variable Selection in Quantile Varying Coefficient Models with Longitudinal Data

Yanlin Tang,Huixia Judy Wang,Zhongyi Zhu

DOI: https://doi.org/10.1016/j.csda.2012.07.015

IF: 2.035

2012-01-01

Computational Statistics & Data Analysis

Abstract:In this paper, we develop a new variable selection procedure for quantile varying coefficient models with longitudinal data. The proposed method is based on basis function approximation and a class of group versions of the adaptive LASSO penalty, which penalizes the Lγ norm of the within-group coefficients with γ≥1. We show that with properly chosen adaptive group weights in the penalization, the resulting penalized estimators are consistent in variable selection, and the estimated functional coefficients retain the optimal convergence rate of nonparametric estimators under the true model. We assess the finite sample performance of the proposed procedure by an extensive simulation study, and the analysis of an AIDS data set and a yeast cell-cycle gene expression data set.
Variable selection for generalized varying coefficient models with longitudinal data

Hu Yang,Chaohui Guo,Jing Lv

DOI: https://doi.org/10.1007/s00362-014-0647-x

2014-01-01

Statistical Papers

Abstract:In this paper, we apply the penalized quadratic inference function to perform variable selection and estimation simultaneously for generalized varying coefficient models with longitudinal data. The proposed approach is based on basis function approximations and the group SCAD penalty, which can incorporate information on the correlation structure within the same subject to achieve an efficient estimator. Furthermore, we discuss the asymptotic theory of our proposed procedure under suitable conditions, including consistency in variable selection and the oracle property in estimation. Finally, monte carlo simulations and a real data analysis are conducted to examine the finite sample performance of the proposed procedure.
Variable Selection in Joint Mean and Variance Models

WU Liu-cang,ZHANG Zhong-zhan,XU Deng-ke

DOI: https://doi.org/10.3969/j.issn.1000-6788.2012.08.015

2012-01-01

Systems Engineering - Theory & Practice

Abstract:In many applications,particularly in the econometric area and industrial quality improvement experiments,there is a great need to model the variance.A unified procedure was proposed to simultaneously select significant variables in joint mean and variance models which provide a useful extension of the classical normal regression models.It is further shown that the presented penalized estimator enjoys the consistency and the oracle property.Simulation and practice show that this model and method are useful and effective.
Automatic Variable Selection for High-Dimensional Linear Models with Longitudinal Data

Ruiqin Tian,Liugen Xue

DOI: https://doi.org/10.4236/ojs.2014.41005

2014-01-01

Open Journal of Statistics

Abstract:High-dimensional longitudinal data arise frequently in biomedical and genomic research. It is important to select relevant covariates when the dimension of the parameters diverges as the sample size increases. We consider the problem of variable selection in high-dimensional linear models with longitudinal data. A new variable selection procedure is proposed using the smooth-threshold generalized estimating equation and quadratic inference functions (SGEE-QIF) to incorporate correlation information. The proposed procedure automatically eliminates inactive predictors by setting the corresponding parameters to be zero, and simultaneously estimates the nonzero regression coefficients by solving the SGEE-QIF. The proposed procedure avoids the convex optimization problem and is flexible and easy to implement. We establish the asymptotic properties in a high-dimensional framework where the number of covariates increases as the number of cluster increases. Extensive Monte Carlo simulation studies are conducted to examine the finite sample performance of the proposed variable selection procedure.
Variable Selection in High-Dimensional Double Generalized Linear Models

Dengke Xu,Zhongzhan Zhang,Liucang Wu

DOI: https://doi.org/10.1007/s00362-012-0481-y

2012-01-01

Statistical Papers

Abstract:In this paper we are concerned with the problems of variable selection and estimation in double generalized linear models in which both the mean and the dispersion are allowed to depend on explanatory variables. We propose a maximum penalized pseudo-likelihood method when the number of parameters diverges with the sample size. With appropriate selection of the tuning parameters, the consistency of the variable selection procedure and asymptotic properties of the resulting estimators are established. We also carry out simulation studies and a real data analysis to assess the finite sample performance of the proposed variable selection procedure, showing that the proposed variable selection method works satisfactorily.
Variable selection for semivarying coefficient models via local averaging

Xinyi Qi,Mengjie Liu,Chuanlong Xie,Heng Peng

DOI: https://doi.org/10.1002/sta4.703

2024-06-06

Stat

Abstract:This study aims to provide novel insights into variable selection in the semivarying coefficient model. We focus on the problem of variable selection and screening for the constant coefficient part. A common approach in the existing literature is to infer the constant coefficients by transforming the problem into a linear model scenario, utilizing a fine estimator of the varying coefficients. In this paper, we propose an approximation method for the varying coefficient functions using local averaging, which is characterized by its simplicity, rough and computational efficiency. Additionally, we introduce an adaptive lasso estimator and a forward regression algorithm specifically designed for semivarying coefficient models. Theoretical and experimental results highlight the effectiveness of the local averaging method in extending variable selection techniques from the linear model to the semivarying coefficient model. Our proposed approaches demonstrate a significant improvement in inference speed compared with baseline methods, with little loss of asymptotic efficiency.

statistics & probability
Variable Selection Using Nonlocal Priors in High-Dimensional Generalized Linear Models With Application to fMRI Data Analysis

Xuan Cao,Kyoungjae Lee

DOI: https://doi.org/10.3390/e22080807

IF: 2.738

2020-07-23

Entropy

Abstract:High-dimensional variable selection is an important research topic in modern statistics. While methods using nonlocal priors have been thoroughly studied for variable selection in linear regression, the crucial high-dimensional model selection properties for nonlocal priors in generalized linear models have not been investigated. In this paper, we consider a hierarchical generalized linear regression model with the product moment nonlocal prior over coefficients and examine its properties. Under standard regularity assumptions, we establish strong model selection consistency in a high-dimensional setting, where the number of covariates is allowed to increase at a sub-exponential rate with the sample size. The Laplace approximation is implemented for computing the posterior probabilities and the shotgun stochastic search procedure is suggested for exploring the posterior space. The proposed method is validated through simulation studies and illustrated by a real data example on functional activity analysis in fMRI study for predicting Parkinson’s disease.

physics, multidisciplinary
Robust Variable Selection for Single-Index Varying-Coefficient Model with Missing Data in Covariates

Yunquan Song,Yaqi Liu,Hang Su

DOI: https://doi.org/10.3390/math10122003

IF: 2.4

2022-01-01

Mathematics

Abstract:As applied sciences grow by leaps and bounds, semiparametric regression analyses have broad applications in various fields, such as engineering, finance, medicine, and public health. Single-index varying-coefficient model is a common class of semiparametric models due to its flexibility and ease of interpretation. The standard single-index varying-coefficient regression models consist mainly of parametric regression and semiparametric regression, which assume that all covariates can be observed. The assumptions are relaxed by taking the models with missing covariates into consideration. To eliminate the possibility of bias due to missing data, we propose a probability weighted objective function. In this paper, we investigate the robust variable selection for a single-index varying-coefficient model with missing covariates. Using parametric and nonparametric estimates of the likelihood of observations with fully observed covariates, we examine the estimators for estimating the likelihood of observations. For variable selection, we use a weighted objective function penalized by a non-convex SCAD. Theoretical challenges include the treatment of missing data and a single-index varying-coefficient model that uses both the non-smooth loss function and the non-convex penalty function. We provide Monte Carlo simulations to evaluate the performance of our approach.

Variable selection in high-dimensional varying-coefficient models with global optimality

Variable Selection for Generalized Varying Coefficient Partially Linear Models with Diverging Number of Parameters

Variable selection in partially time-varying coefficient models

A Unified Variable Selection Approach for Varying Coefficient Models

Variable Selection for High Dimensional Partially Linear Varying Coefficient Errors-in-variables Models

Variable Selection in High-Dimensional Quantile Varying Coefficient Models

Automatic Variable Selection for Varying Coefficient Models with Longitudinal Data

A New Variable Selection Approach for Varying Coefficient Models

Structural identification and variable selection in high-dimensional varying-coefficient models

Variable selection for partially linear single-index varying-coefficient model

Variable Selection in High-dimensional Partially Linear Models

Robust exponential squared loss-based variable selection for high-dimensional single-index varying-coefficient model

Variable Selection for Fixed Effects Varying Coefficient Models

Variable Selection in Quantile Varying Coefficient Models with Longitudinal Data

Variable selection for generalized varying coefficient models with longitudinal data

Variable Selection in Joint Mean and Variance Models

Automatic Variable Selection for High-Dimensional Linear Models with Longitudinal Data

Variable Selection in High-Dimensional Double Generalized Linear Models

Variable selection for semivarying coefficient models via local averaging

Variable Selection Using Nonlocal Priors in High-Dimensional Generalized Linear Models With Application to fMRI Data Analysis

Robust Variable Selection for Single-Index Varying-Coefficient Model with Missing Data in Covariates