Abstract:When fitting statistical models, some predictors are often found to be correlated with each other, and functioning together. Many group variable selection methods are developed to select the groups of predictors that are closely related to the continuous or categorical response. These existing methods usually assume the group structures are well known. For example, variables with similar practical meaning, or dummy variables created by categorical data. However, in practice, it is impractical to know the exact group structure, especially when the variable dimensional is large. As a result, the group variable selection results may be selected. To solve the challenge, we propose a two-stage approach that combines a variable clustering stage and a group variable stage for the group variable selection problem. The variable clustering stage uses information from the data to find a group structure, which improves the performance of the existing group variable selection methods. For ultrahigh dimensional data, where the predictors are much larger than observations, we incorporated a variable screening method in the first stage and shows the advantages of such an approach. In this article, we compared and discussed the performance of four existing group variable selection methods under different simulation models, with and without the variable clustering stage. The two-stage method shows a better performance, in terms of the prediction accuracy, as well as in the accuracy to select active predictors. An athlete's data is also used to show the advantages of the proposed method.

Factor Profiling for Ultra High Dimensional Variable Selection

Factor Profiled Sure Independence Screening

Robust variable screening for regression using factor profiling

Profiled Forward Regression for Ultrahigh Dimensional Variable Screening in Semiparametric Partially Linear Models

Variable screening using factor analysis for high-dimensional data with multicollinearity

Profile forward regression screening for ultra-high dimensional semiparametric varying coefficient partially linear models

Forward Regression for Ultra-High Dimensional Variable Screening

High-Dimensional Variable Screening Under Multicollinearity

Ultrahigh dimensional variable selection: beyond the linear model

A Two-Stage Variable Selection Approach for Correlated High Dimensional Predictors

Factor models and variable selection in high-dimensional regression analysis

Estimation for ultra-high dimensional factor model: a pivotal variable detection based approach

Feature Screening with Latent Responses

Projected principal component analysis in factor models

A robust variable screening procedure for ultra-high dimensional data

High-dimensional Ordinary Least-squares Projection for Screening Variables

Profile Greedy Forward Regression Variable Screening for Ultra-High Dimensional Partially Linear Model

Deep Feature Screening: Feature Selection for Ultra High-Dimensional Data via Deep Neural Networks

Dynamic Tilted Current Correlation for High Dimensional Variable Screening

Feature Screening for Multi-Response Varying Coefficient Models with Ultrahigh Dimensional Predictors

Sequential profile Lasso for ultra-high-dimensional partially linear models