Abstract:Predictive modeling has a wide range of applications in HEOR. When the number of predictors is large in comparison to the number of observations, conventional generalized linear regression models can yield poor predictions. Machine Learning (ML) methods are increasingly being used in HEOR as an alternative framework to increase prediction accuracy. Commonly used ML models include Logistic-LASSO, tree based models (CART, Random Forest), and Neural Networks. This research aimed to explore the value of ML for predictive modeling in HEOR, using high dimensional datasets like genomic or claims, and the tradeoffs between model performance and interpretability. Although ML models often outperform regression-based approaches in prediction quality, questions remain about their interpretability. Comparisons between ML and conventional regression models were performed in three different settings: High Dimensional Propensity Score (HDPS) to estimate treatment effect, predicting treatment resistance using genomic data, and classifying patients for risk of an undiagnosed condition. Predictive performance of ML models including Logistic-LASSO, Random Forests, Neural Networks and a conventional logistic regression model were evaluated both in-sample and out-of-sample. Prediction accuracy (i.e. ROC-AUC), and bias (through simulations) were assessed for the different models. In each application, cross validation was used to minimize overfitting risk. ML models outperformed regression-based approaches in terms of prediction accuracy (increased ROC-AUC by ≥10%) and significantly decreased estimation bias (15% lower than regression in the HDPS application). However, there was a significant loss of interpretability, especially with Deep Neural Networks and Random Forests. Quantifying the impact of specific predictors on outcome is virtually impossible in these instances. ML models exhibited better predictive performance measured by ROC-AUC and bias than conventional logistic regression models at the expense of interpretability. Conventional regression models can provide a better framework if interpretability is central to the prediction analysis.

High-Dimensional Propensity Score and Its Machine Learning Extensions in Residual Confounding Control

High-dimensional propensity scores for empirical covariate selection in secondary database studies: Planning, implementation, and reporting

Propensity score prediction for electronic healthcare databases using super learner and high-dimensional propensity score methods

Deep Learning-based Propensity Scores for Confounding Control in Comparative Effectiveness Research A Large-scale, Real-world Data Study

Adaptability of High Dimensional Propensity Score Procedure in the Transition from ICD-9 to ICD-10 in the US Healthcare System

PNS196 USE OF MACHINE LEARNING IN HEALTH ECONOMICS AND OUTCOMES RESEARCH (HEOR): OPTIMIZING PREDICTIVE MODELS

Propensity scores using missingness pattern information: a practical guide

Propensity Score Method: a Non-Parametric Technique to Reduce Model Dependence

Propensity score analysis for time-dependent exposure

Bayesian propensity scores for high-dimensional causal inference: A comparison of drug-eluting to bare-metal coronary stents

Use of Propensity Scoring and Its Application to Real‐World Data: Advantages, Disadvantages, and Methodological Objectives Explained to Researchers Without Using Mathematical Equations

Deep propensity network using a sparse autoencoder for estimation of treatment effects

1290-P: Developing Machine Learning Model for Predicting Acute Coronary Syndrome in Type 2 Diabetes Mellitus Patients through Substitution of Propensity Scores for Binary Variables

Collaborative-controlled LASSO for Constructing Propensity Score-based Estimators in High-Dimensional Data

High-dimensional propensity score adjustment in HIV research using linked administrative health data

Targeted Learning with an Undersmoothed Lasso Propensity Score Model for Large-Scale Covariate Adjustment in Healthcare Database Studies

Estimating Treatment Effect under Additive Hazards Models with High-dimensional Covariates

Using permutations to assess confounding in machine learning applications for digital health

Estimating and Using Propensity Scores with Partially Missing Data

High-dimensional mediation analysis for continuous outcome with confounders using overlap weighting method in observational epigenetic study

High Dimensional Propensity Score Estimation via Covariate Balancing