Abstract:Evidence for Action (E4A), a signature program of the Robert Wood Johnson Foundation, funds investigator-initiated research on the impacts of social programs and policies on population health and health inequities. Across thousands of letters of intent and full proposals E4A has received since 2015, one of the most common methodological challenges faced by applicants is selecting realistic effect sizes to inform power and sample size calculations. E4A prioritizes health studies that are both (1) adequately powered to detect effect sizes that may reasonably be expected for the given intervention and (2) likely to achieve intervention effects sizes that, if demonstrated, correspond to actionable evidence for population health stakeholders. However, little guidance exists to inform the selection of effect sizes for population health research proposals. We draw on examples of five rigorously evaluated population health interventions. These examples illustrate considerations for selecting realistic and actionable effect sizes as inputs to power and sample size calculations for research proposals to study population health interventions. We show that plausible effects sizes for population health inteventions may be smaller than commonly cited guidelines suggest. Effect sizes achieved with population health interventions depend on the characteristics of the intervention, the target population, and the outcomes studied. Population health impact depends on the proportion of the population receiving the intervention. When adequately powered, even studies of interventions with small effect sizes can offer valuable evidence to inform population health if such interventions can be implemented broadly. Demonstrating the effectiveness of such interventions, however, requires large sample sizes.

An Aggregation Scheme for Increased Power

Increasing power for observational studies of aberrant response: An adaptive approach

Using Power Analysis to Choose the Unit of Randomization, Outcome, and Approach for Subgroup Analysis for a Multilevel Randomized Controlled Clinical Trial to Reduce Disparities in Cardiovascular Health

More Powerful Multiple Testing in Randomized Experiments with Non-Compliance

More power to you: Using machine learning to augment human coding for more efficient inference in text-based randomized trials

Statistical Inference and Power Analysis for Direct and Spillover Effects in Two-Stage Randomized Experiments

Finding the right power balance: Better study design and collaboration can reduce dependence on statistical power

Improving the Power of the Randomization Test

Asymptotic Results on a Class of Adaptive Multi-Treatment Designs

Organizational Effectiveness: A New Strategy to Leverage Multisite Randomized Trials for Valid Assessment

Power Under Multiplicity Project (PUMP): Estimating Power, Minimum Detectable Effect Size, and Sample Size When Adjusting for Multiple Outcomes in Multi-level Experiments

A Potential Outcomes, and Typically More Powerful, Alternative to 'Cochran-Mantel-Haenszel'

Power and Sample Size Calculations for Rerandomization

Power considerations for generalized estimating equations analyses of four-level cluster randomized trials

Propensity Score Methods for Merging Observational and Experimental Datasets

Calculating the power of a planned individual participant data meta‐analysis to examine prognostic factor effects for a binary outcome

Augmenting treatment arms with external data through propensity-score weighted power-priors: an application in expanded access

Power Analysis for Cluster Randomized Trials with Binary Outcomes Modeled by Generalized Linear Mixed-Effects Models

Evidence Aggregation for Treatment Choice

Powering population health research: Considerations for plausible and actionable effect sizes

Estimating causal effects for binary outcomes using per-decision inverse probability weighting