Who Are We Missing? A Principled Approach to Characterizing the Underrepresented Population

Harsh Parikh,Rachael Ross,Elizabeth Stuart,Kara Rudolph

2024-08-26

Abstract:Randomized controlled trials (RCTs) serve as the cornerstone for understanding causal effects, yet extending inferences to target populations presents challenges due to effect heterogeneity and underrepresentation. Our paper addresses the critical issue of identifying and characterizing underrepresented subgroups in RCTs, proposing a novel framework for refining target populations to improve generalizability. We introduce an optimization-based approach, Rashomon Set of Optimal Trees (ROOT), to characterize underrepresented groups. ROOT optimizes the target subpopulation distribution by minimizing the variance of the target average treatment effect estimate, ensuring more precise treatment effect estimations. Notably, ROOT generates interpretable characteristics of the underrepresented population, aiding researchers in effective communication. Our approach demonstrates improved precision and interpretability compared to alternatives, as illustrated with synthetic data experiments. We apply our methodology to extend inferences from the Starting Treatment with Agonist Replacement Therapies (START) trial -- investigating the effectiveness of medication for opioid use disorder -- to the real-world population represented by the Treatment Episode Dataset: Admissions (TEDS-A). By refining target populations using ROOT, our framework offers a systematic approach to enhance decision-making accuracy and inform future trials in diverse populations.

Methodology,Computers and Society,Machine Learning,Applications

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the target population inference problem in randomized controlled trials (RCTs). Specifically, although RCTs have internal validity in estimating treatment effects, due to effect heterogeneity and the under - representation of certain sub - groups in the trials, there are challenges in generalizing their results to the target population. The paper proposes a new framework aimed at identifying and characterizing under - represented sub - groups in RCTs to improve the universality of inference. To this end, the authors introduce an optimization - based method - Rashomon Set of Optimal Trees (ROOT) - to characterize these under - represented groups. By minimizing the variance of the target average treatment effect estimate, ROOT not only improves the precision of the treatment effect estimate but also generates interpretable features of these groups, which helps researchers communicate research results effectively.

The key contributions of the paper are:
1. **Identifying Challenging Sub - groups**: A method is proposed to identify sub - groups for which treatment effects are difficult to estimate precisely. These sub - groups are usually located in regions of the covariate space with high effect heterogeneity and insufficient data support.
2. **Optimizing the Target Population Distribution**: By optimizing the distribution of the target sub - population, the variance of the target average treatment effect estimate is minimized, ensuring more precise treatment effect estimates.
3. **Improving Interpretability**: Interpretable features of under - represented groups are generated, helping researchers better understand which populations can be inferred with high confidence.
4. **Application Example**: The method is applied to the Starting Treatment with Agonist Replacement Therapies (START) trial to explore how to generalize the trial results to the real - world population represented by Treatment Episode Dataset: Admissions (TEDS - A).

Through this method, the paper provides a systematic framework to enhance the accuracy of decision - making and inform future trials in diverse populations.