Abstract:In variable selection, a selection rule that prescribes the permissible sets of selected variables (called a "selection dictionary") is desirable due to the inherent structural constraints among the candidate variables. Such selection rules can be complex in real-world data analyses, and failing to incorporate such restrictions could not only compromise the interpretability of the model but also lead to decreased prediction accuracy. However, no general framework has been proposed to formalize selection rules and their applications, which poses a significant challenge for practitioners seeking to integrate these rules into their analyses. In this work, we establish a framework for structured variable selection that can incorporate universal structural constraints. We develop a mathematical language for constructing arbitrary selection rules, where the selection dictionary is formally defined. We demonstrate that all selection rules can be expressed as combinations of operations on constructs, facilitating the identification of the corresponding selection dictionary. Once this selection dictionary is derived, practitioners can apply their own user-defined criteria to select the optimal model. Additionally, our framework enhances existing penalized regression methods for variable selection by providing guidance on how to appropriately group variables to achieve the desired selection rule. Furthermore, our innovative framework opens the door to establishing new l0 norm-based penalized regression techniques that can be tailored to respect arbitrary selection rules, thereby expanding the possibilities for more robust and tailored model development.

Budget constrained model selection for multiple linear regression

Subset selection for multiple linear regression via optimization

A mathematical programming approach for integrated multiple linear regression subset selection and validation

An efficient optimization approach for best subset selection in linear regression, with application to model selection and fitting in autoregressive time-series

Efficient Simulation Budget Allocation for Subset Selection Using Regression Metamodels

A Unified Variable Selection Approach for Varying Coefficient Models

Cost-sensitive selection of variables by ensemble of model sequences

Selecting an Optimal Subset with Regression Metamodels

Study of Bayesian variable selection method on mixed linear regression models

Model selection and estimation in regression with grouped variables

Selection of Regression Models under Linear Restrictions for Fixed and Random Designs

Variable Selection for Linear Mixed Effects Model Via Penalization Approaches

A LASSO-Type Approach to Variable Selection and Estimation for Censored Regression Model

Simultaneous Structure Estimation And Variable Selection In Partial Linear Varying Coefficient Models For Longitudinal Data

Economic variable selection

Analysis of Longitudinal Data Using Varying-Coefficient Model

Variable Selection in Robust Joint Mean and Covariance Model for Longitudinal Data Analysis

New Robust Variable Selection Methods for Linear Regression Models

A general framework for formulating structured variable selection

Variable Selection for Multiple Function-on-Function Linear Regression

Variable Selection of Spatial Logistic Autoregressive Model with Linear Constraints