Abstract:When fitting statistical models, some predictors are often found to be correlated with each other, and functioning together. Many group variable selection methods are developed to select the groups of predictors that are closely related to the continuous or categorical response. These existing methods usually assume the group structures are well known. For example, variables with similar practical meaning, or dummy variables created by categorical data. However, in practice, it is impractical to know the exact group structure, especially when the variable dimensional is large. As a result, the group variable selection results may be selected. To solve the challenge, we propose a two-stage approach that combines a variable clustering stage and a group variable stage for the group variable selection problem. The variable clustering stage uses information from the data to find a group structure, which improves the performance of the existing group variable selection methods. For ultrahigh dimensional data, where the predictors are much larger than observations, we incorporated a variable screening method in the first stage and shows the advantages of such an approach. In this article, we compared and discussed the performance of four existing group variable selection methods under different simulation models, with and without the variable clustering stage. The two-stage method shows a better performance, in terms of the prediction accuracy, as well as in the accuracy to select active predictors. An athlete's data is also used to show the advantages of the proposed method.

High-dimensional variable selection via tilting

Faithful Variable Screening for High-Dimensional Convex Regression

Dynamic Tilted Current Correlation for High Dimensional Variable Screening

Variable screening in multivariate linear regression with high-dimensional covariates

Ultrahigh dimensional variable selection: beyond the linear model

Variable Selection in High-Dimensional Quantile Varying Coefficient Models

A Selective Overview of Variable Selection in High Dimensional Feature Space (Invited Review Article)

Bayesian variable selection in linear regression models with instrumental variables

Quantile Correlation-based Variable Selection

Variable Selection for High Dimensional Partially Linear Varying Coefficient Errors-in-variables Models

Consistent Tuning Parameter Selection in High Dimensional Sparse Linear Regression

Variable Selection in High-Dimensional Error-in-Variables Models Via Controlling the False Discovery Proportion

Variable Selection for High Dimensional Gaussian Copula Regression Model: an Adaptive Hypothesis Testing Procedure.

Screen then select: a strategy for correlated predictors in high-dimensional quantile regression

Automatic Variable Selection for High-Dimensional Linear Models with Longitudinal Data

Variable Selection For Ultra-High Dimensional Quantile Regression With Missing Data And Measurement Error

Structural identification and variable selection in high-dimensional varying-coefficient models

Variable Selection in High-Dimensional Sparse Multiresponse Linear Regression Models

A Two-Stage Variable Selection Approach for Correlated High Dimensional Predictors

A Transparent and Nonlinear Method for Variable Selection

Forward Variable Selection for Sparse Ultra-High Dimensional Varying Coefficient Models