Abstract:When fitting statistical models, some predictors are often found to be correlated with each other, and functioning together. Many group variable selection methods are developed to select the groups of predictors that are closely related to the continuous or categorical response. These existing methods usually assume the group structures are well known. For example, variables with similar practical meaning, or dummy variables created by categorical data. However, in practice, it is impractical to know the exact group structure, especially when the variable dimensional is large. As a result, the group variable selection results may be selected. To solve the challenge, we propose a two-stage approach that combines a variable clustering stage and a group variable stage for the group variable selection problem. The variable clustering stage uses information from the data to find a group structure, which improves the performance of the existing group variable selection methods. For ultrahigh dimensional data, where the predictors are much larger than observations, we incorporated a variable screening method in the first stage and shows the advantages of such an approach. In this article, we compared and discussed the performance of four existing group variable selection methods under different simulation models, with and without the variable clustering stage. The two-stage method shows a better performance, in terms of the prediction accuracy, as well as in the accuracy to select active predictors. An athlete's data is also used to show the advantages of the proposed method.

Variable selection in discriminant analysis based on Gram-Schmidt process

Variable Selection in Discriminant Partial Least-Squares Analysis

Variable Selection for Generalized Varying Coefficient Partially Linear Models with Diverging Number of Parameters

Dimension Reduction Method of Support Vector Machine Based on Gram-Schmidt Process

Variable selection in model-based clustering and discriminant analysis with a regularization approach

An Efficient Variable Selection Method for Predictive Discriminant Analysis

Optimal Feature Selection in High-Dimensional Discriminant Analysis

Main and Interaction Effects Selection for Quadratic Discriminant Analysis via Penalized Linear Regression

On Bic'S Selection Consistency for Discriminant Analysis

Variational Discriminant Analysis with Variable Selection

Effect Size Estimation and Misclassification Rate Based Variable Selection in Linear Discriminant Analysis

Functional variable selection via Gram–Schmidt orthogonalization for multiple functional linear regression

Variable Selection in Logistic Regression Model with Genetic Algorithm.

Optimal Feature Selection for Sparse Linear Discriminant Analysis and Its Applications in Gene Expression Data

Variable Selection Based on Schmidt Process and Its Application in Forestry Coverage Analysis

Variable Selection in High-Dimensional Error-in-Variables Models Via Controlling the False Discovery Proportion

A method for variable selection in a multivariate functional linear regression model

Variable selection and updating in model-based discriminant analysis for high dimensional data with food authenticity applications

Nonstationary Gaussian Process Discriminant Analysis With Variable Selection for High-Dimensional Functional Data

Robust and Efficient Linear Discriminant Analysis With L2, 1-Norm for Feature Selection

A Two-Stage Variable Selection Approach for Correlated High Dimensional Predictors