Abstract:<section class="article-section__content"><h3 class="article-section__sub-title section1"> Objective</h3><p>To define confounding bias in difference‐in‐difference studies and compare regression‐ and matching‐based estimators designed to correct bias due to observed confounders.</p></section><section class="article-section__content"><h3 class="article-section__sub-title section1"> Data sources</h3><p>We simulated data from linear models that incorporated different confounding relationships: time‐invariant covariates with a time‐varying effect on the outcome, time‐varying covariates with a constant effect on the outcome, and time‐varying covariates with a time‐varying effect on the outcome. We considered a simple setting that is common in the applied literature: treatment is introduced at a single time point and there is no unobserved treatment effect heterogeneity.</p></section><section class="article-section__content"><h3 class="article-section__sub-title section1"> Study design</h3><p>We compared the bias and root mean squared error of treatment effect estimates from six model specifications, including simple linear regression models and matching techniques.</p></section><section class="article-section__content"><h3 class="article-section__sub-title section1"> Data collection</h3><p>Simulation code is provided for replication.</p></section><section class="article-section__content"><h3 class="article-section__sub-title section1"> Principal findings</h3><p>Confounders in difference‐in‐differences are covariates that change differently over time in the treated and comparison group or have a time‐varying effect on the outcome. When such a confounding variable is measured, appropriately adjusting for this confounder (ie, including the confounder in a regression model that is consistent with the causal model) can provide unbiased estimates with optimal SE. However, when a time‐varying confounder is affected by treatment, recovering an unbiased causal effect using difference‐in‐differences is difficult.</p></section><section class="article-section__content"><h3 class="article-section__sub-title section1"> Conclusions</h3><p>Confounding in difference‐in‐differences is more complicated than in cross‐sectional settings, from which techniques and intuition to address observed confounding cannot be imported wholesale. Instead, analysts should begin by postulating a causal model that relates covariates, both time‐varying and those with time‐varying effects on the outcome, to treatment. This causal model will then guide the specification of an appropriate analytical model (eg, using regression or matching) that can produce unbiased treatment effect estimates. We emphasize the importance of thoughtful incorporation of covariates to address confounding bias in difference‐in‐difference studies.</p></section>

Confounding, Homogeneity and Collapsibility for Causal Effects in Epidemiologic Studies

Detecting multiple confounders

Criteria for Confounders in Epidemiological Studies

Conditions for Non-confounding and Collapsibility without Knowledge of Completely Constructed Causal Diagrams

Distinguishing Between Mediators and Confounders is Important for the Causal Inference in Observational Studies

Confounding and Confounding Control

Sensitivity Analysis for Unmeasured Confounding in the Estimation of Marginal Causal Effects

Detecting and Measuring Confounding Using Causal Mechanism Shifts

Rothman diagrams: the geometry of causal inference in epidemiology

Confounding caused by causal-effect covariability

Population Heterogeneity and Causal Inference

Identification of confounder in epidemiologic data contaminated by measurement error in covariates

Confounding and regression adjustment in difference‐in‐differences studies

Effect heterogeneity and variable selection for standardizing causal effects to a target population

Several untestable assumptions in epidemiological observational studies: control models and mechanism of missing data

Individual causal effects from observational longitudinal studies with time-varying exposures

On Heterogeneous Treatment Effects in Heterogeneous Causal Graphs

Identifiability, Stratification and Minimum Variance Estimation of Causal Effects

A Comparison of Approaches to Control for Confounding Factors by Regression Models

The built-in selection bias of hazard ratios formalized using structural causal models

Rothman diagrams: the geometry of confounding and standardization