Abstract:The missing data issue is ubiquitous in health studies. Variable selection in the presence of both missing covariates and outcomes is an important statistical research topic but has been less studied. Existing literature focuses on parametric regression techniques that provide direct parameter estimates of the regression model. Flexible nonparametric machine learning methods considerably mitigate the reliance on the parametric assumptions, but do not provide as naturally defined variable importance measure as the covariate effect native to parametric models. We investigate a general variable selection approach when both the covariates and outcomes can be missing at random and have general missing data patterns. This approach exploits the flexibility of machine learning modeling techniques and bootstrap imputation, which is amenable to nonparametric methods in which the covariate effects are not directly available. We conduct expansive simulations investigating the practical operating characteristics of the proposed variable selection approach, when combined with four tree-based machine learning methods, XGBoost, Random Forests, Bayesian Additive Regression Trees (BART) and Conditional Random Forests, and two commonly used parametric methods, lasso and backward stepwise selection. Numeric results suggest that when combined with bootstrap imputation, XGBoost and BART have the overall best variable selection performance with respect to the $F_1$ score and Type I error across various settings. In general, there is no significant difference in the variable selection performance due to imputation methods. We further demonstrate the methods via a case study of risk factors for 3-year incidence of metabolic syndrome with data from the Study of Women's Health Across the Nation.

Instability of Variable-selection Algorithms Used to Identify True Predictors of an Outcome in Intermediate-dimension Epidemiologic Studies

Variable selection for competing risk regression models: recommendations for analyzing data from epidemiological studies

Stability Selection for Structured Variable Selection

Stable variable selection for right censored data: comparison of methods

Trimming Stability Selection increases variable selection robustness

Stability of clinical prediction models developed using statistical or machine learning methods

Nonparametric Assessment of Variable Selection and Ranking Algorithms

Variable selection with missing data in both covariates and outcomes: Imputation and machine learning

Comparing Stochastic Optimization Methods for Variable Selection in Binary Outcome Prediction, With Application to Health Policy

A Computational Exploration of Emerging Methods of Variable Importance Estimation

Stable predictions for health related anticausal prediction tasks affected by selection biases: the need to deconfound the test set features

High-dimensional Feature Selection in Competing Risks Modeling: A Stable Approach Using a Split-and-merge Ensemble Algorithm

Ensembling Variable Selectors by Stability Selection for the Cox Model

Interpretability of bi‐level variable selection methods

Cluster Stability Selection

Efficient and Doubly-Robust Methods for Variable Selection and Parameter Estimation in Longitudinal Data Analysis

New Algorithms and Software for Significance Controlled Variable Selection

Variable Selection in Rank Regression for Analyzing Longitudinal Data

Loss-guided Stability Selection

Irreproducibility; Nothing is More Predictable

On the variability of regression shrinkage methods for clinical prediction models: simulation study on predictive performance