Abstract:Traditional variable-selection strategies in generalized linear models (GLMs) seek to optimize a measure of predictive accuracy without regard for the cost of data collection. When the purpose of such model building is the creation of predictive scales to be used in future studies with constrained budgets, the standard approach may not be optimal. We propose a Bayesian decision-theoretic framework for variable selection in binary-outcome GLMs where the budget for data collection is constrained and potential predictors may vary considerably in cost. The method is illustrated using data from a large study of quality of hospital care in the U.S. in the 1980s. Especially when the number of available predictors p is large, it is important to use an appropriate technique for optimization (e.g., in an application presented here where p = 83, the space over which we search has 283 ≐ 1025 elements, which is too large to explore using brute force enumeration). Specifically, we investigate simulated annealing (SA), genetic algorithms (GAs), and the tabu search (TS) method used in operations research, and we develop a context-specific version of SA, improved simulated annealing (ISA), that performs better than its generic counterpart. When p was modest in our study, we found that GAs performed relatively poorly for all but the very best user-defined input configurations, generic SA did not perform well, and TS had excellent median performance and was much less sensitive to suboptimal choice of user-defined inputs. When p was large in our study, the best versions of GA and ISA outperformed TS and generic SA. Our results are presented in the context of health policy but can apply to other quality assessment settings with dichotomous outcomes as well.

Old but Gold or New and Shiny? Comparing Tree Ensembles for Ordinal Prediction with a Classic Parametric Approach

Frequency‐adjusted borders ordinal forest: A novel tree ensemble method for ordinal prediction

Ordinal Trees and Random Forests: Score-Free Recursive Partitioning and Improved Ensembles

A Tree-Based Method for Modeling a Multivariate Ordinal Response.

Splitting criteria for ordinal decision trees: an experimental study

Rank Entropy-Based Decision Trees for Monotonic Classification

Hypothesis Tests in Ordinal Predictive Models with Optimal Accuracy.

Comparing Stochastic Optimization Methods for Variable Selection in Binary Outcome Prediction, With Application to Health Policy

Ranking Perspective for Tree-based Methods with Applications to Symbolic Feature Selection

Transductive Ordinal Regression

Optimal or Greedy Decision Trees? Revisiting their Objectives, Tuning, and Performance

Deep and interpretable regression models for ordinal outcomes

Policy Trees for Prediction: Interpretable and Adaptive Model Selection for Machine Learning

Optimal Decision Tree and Adaptive Submodular Ranking with Noisy Outcomes

Deck of Cards method for Hierarchical, Robust and Stochastic Ordinal Regression

Ordered probit Bayesian additive regression trees for ordinal data

Born-Again Tree Ensembles

On Computing Optimal Tree Ensembles

An exploratory factor model for ordinal paired comparison indicators

Building Trees for Probabilistic Prediction via Scoring Rules

Supervised Feature Subset Selection with Ordinal Optimization