Abstract:Researchers frequently employ difference-in-differences (DiD) to study the impact of public health interventions on infectious disease outcomes. DiD assumes that treatment and non-experimental comparison groups would have moved in parallel in expectation, absent the intervention (“parallel trends assumption”). However, the plausibility of parallel trends assumption in the context of infectious disease transmission is not well-understood. Our work bridges this gap by formalizing epidemiological assumptions required for common DiD specifications, positing an underlying Susceptible-Infectious-Recovered (SIR) data-generating process. We demonstrate that popular specifications can encode strict epidemiological assumptions. For example, DiD modeling incident case numbers or rates as outcomes will produce biased treatment effect estimates unless untreated potential outcomes for treatment and comparison groups come from a data-generating process with the same initial infection and equal transmission rates at each time step. Applying a log transformation or modeling log growth allows for different initial infection rates under an “infinite susceptible population” assumption, but invokes conditions on transmission parameters. We then propose alternative DiD specifications based on epidemiological parameters – the effective reproduction number and the effective contact rate – that are both more robust to differences between treatment and comparison groups and can be extended to complex transmission dynamics. With minimal power difference incidence and log incidence models, we recommend a default of the more robust log specification. Our alternative specifications have lower power than incidence or log incidence models, but have higher power than log growth models. We illustrate implications of our work by re-analyzing published studies of COVID-19 mask policies.

Understanding Difference-in-differences methods to evaluate policy effects with staggered adoption: an application to Medicaid and HIV

Designing Difference-in-Difference Studies with Staggered Treatment Adoption: Key Concepts and Practical Guidelines

Difference-in-Differences for Health Policy and Practice: A Review of Modern Methods

A Generalized Difference-in-Differences Estimator for Randomized Stepped-Wedge and Observational Staggered Adoption Settings

Parallel Trends in an Unparalleled Pandemic Difference-in-differences for infectious disease policy evaluation

A Semiparametric Instrumented Difference-in-Differences Approach to Policy Learning

Identifying and estimating effects of sustained interventions under parallel trends assumptions

Making Sense of the Difference-in-Difference Design

Eliminating Systematic Bias from Difference-in-Differences Design: A Permutational Detrending Strategy

Identifying Optimal Methods for Addressing Confounding Bias When Estimating the Effects of State-Level Policies

Difference-in-Differences for Policy Evaluation

Difference-in-Differences with Unpoolable Data

Revisiting Event-Study Designs: Robust and Efficient Estimation

Triangulating Instrumental Variable, confounder adjustment and Difference-in-Difference methods for comparative effectiveness research in observational data

Privacy-preserving impact evaluation using Difference-in-Differences

Bayesian variable selection in hierarchical difference-in-differences models

Fuzzy Differences-in-Differences

Statistical Power for Estimating Treatment Effects Using Difference-in-Differences and Comparative Interrupted Time Series Estimators With Variation in Treatment Timing

Staggered interventions with no control groups

Simple Approaches to Nonlinear Difference-in-Differences with Panel Data

Causal Duration Analysis with Diff-in-Diff