Abstract:Traditional variable-selection strategies in generalized linear models (GLMs) seek to optimize a measure of predictive accuracy without regard for the cost of data collection. When the purpose of such model building is the creation of predictive scales to be used in future studies with constrained budgets, the standard approach may not be optimal. We propose a Bayesian decision-theoretic framework for variable selection in binary-outcome GLMs where the budget for data collection is constrained and potential predictors may vary considerably in cost. The method is illustrated using data from a large study of quality of hospital care in the U.S. in the 1980s. Especially when the number of available predictors p is large, it is important to use an appropriate technique for optimization (e.g., in an application presented here where p = 83, the space over which we search has 283 ≐ 1025 elements, which is too large to explore using brute force enumeration). Specifically, we investigate simulated annealing (SA), genetic algorithms (GAs), and the tabu search (TS) method used in operations research, and we develop a context-specific version of SA, improved simulated annealing (ISA), that performs better than its generic counterpart. When p was modest in our study, we found that GAs performed relatively poorly for all but the very best user-defined input configurations, generic SA did not perform well, and TS had excellent median performance and was much less sensitive to suboptimal choice of user-defined inputs. When p was large in our study, the best versions of GA and ISA outperformed TS and generic SA. Our results are presented in the context of health policy but can apply to other quality assessment settings with dichotomous outcomes as well.

Exploring Hoover and Perez's experimental designs using global sensitivity analysis

Global Sensitivity Analysis for Optimization with Variable Selection

A New Methodology Based on Covariance and Hdmr for Global Sensitivity Analysis

A Global Sensitivity Analysis Method Based on ANOVA with Convex Set Model and Its State-Dependent Parameter Solution

A Two-Stage Method for the Estimation of Global Sensitivity Indices of Non-Parametric Models

A screening approach for non-parametric global sensitivity analysis

Variance-based sensitivity analysis: The quest for better estimators and designs between explorativity and economy

A-optimal designs for non-parametric symmetrical global sensitivity analysis

Design of Experiment in Global Sensitivity Analysis Based on ANOVA High-Dimensional Model Representation

Comparing Stochastic Optimization Methods for Variable Selection in Binary Outcome Prediction, With Application to Health Policy

Variance-based sensitivity analysis for time-dependent processes

Nonparametric Assessment of Variable Selection and Ranking Algorithms

Huber Loss Meets Spatial Autoregressive Model: A Robust Variable Selection Method with Prior Information

Enhancing Variable Importance in Random Forests: A Novel Application of Global Sensitivity Analysis

Adaptively Sampling via Regional Variance-Based Sensitivities

Variable Selection in Macroeconomic Forecasting with Many Predictors

Variable Selection Based Testing for Parameter Changes in Regression with Autoregressive Dependence

A new paradigm for global sensitivity analysis

Sensitivity Analysis of G-estimators to Invalid Instrumental Variables

Investigating the potential of Morris algorithm for improving the computational constraints of global sensitivity analysis

Extended Monte Carlo Simulation for Parametric Global Sensitivity Analysis and Optimization