Abstract:Causal effects are commonly defined as comparisons of the potential outcomes under treatment and control, but this definition is threatened by the possibility that either the treatment or the control condition is not well defined, existing instead in more than one version. This is often a real possibility in nonexperimental or observational studies of treatments because these treatments occur in the natural or social world without the laboratory control needed to ensure identically the same treatment or control condition occurs in every instance. We consider the simplest case: Either the treatment condition or the control condition exists in two versions that are easily recognized in the data but are of uncertain, perhaps doubtful, relevance, for example, branded Advil versus generic ibuprofen. Common practice does not address versions of treatment: Typically, the issue is either ignored or explicitly stated but assumed to be absent. Common practice is reluctant to address two versions of treatment because the obvious solution entails dividing the data into two parts with two analyses, thereby (a) reducing power to detect versions of treatment in each part, (b) creating problems of multiple inference in coordinating the two analyses, and (c) failing to report a single primary analysis that uses everyone. We propose and illustrate a new method of analysis that begins with a single primary analysis of everyone that would be correct if the two versions do not differ, adds a second analysis that would be correct were there two different effects for the two versions, controls the family-wise error rate in all assertions made by the several analyses, and yet pays no price in power to detect a constant treatment effect in the primary analysis of everyone. Our method can be applied to analyses of constant additive treatment effects on continuous outcomes. Unlike conventional simultaneous inferences, the new method is coordinating several analyses that are valid under different assumptions, so that one analysis would never be performed if one knew for certain that the assumptions of the other analysis are true. It is a multiple assumptions problem rather than a multiple hypotheses problem. We discuss the relative merits of the method with respect to more conventional approaches to analyzing multiple comparisons. The method is motivated and illustrated using a study of the possibility that repeated head trauma in high school football causes an increase in risk of early onset cognitive decline.

Making Sense of the Difference-in-Difference Design

Designing Difference-in-Difference Studies with Staggered Treatment Adoption: Key Concepts and Practical Guidelines

Understanding Difference-in-differences methods to evaluate policy effects with staggered adoption: an application to Medicaid and HIV

Difference-in-Differences for Health Policy and Practice: A Review of Modern Methods

Fuzzy Differences-in-Differences

Eliminating Systematic Bias from Difference-in-Differences Design: A Permutational Detrending Strategy

Designing observational studies for credible causal inference in addiction research—Directed acyclic graphs, modified disjunctive cause criterion and target trial emulation

Causal inference with observational data in addiction research

Difference-in-Discontinuities: Estimation, Inference and Validity Tests

Distinguishing Between Mediators and Confounders is Important for the Causal Inference in Observational Studies

Identifying and estimating effects of sustained interventions under parallel trends assumptions

Triangulating Instrumental Variable, confounder adjustment and Difference-in-Difference methods for comparative effectiveness research in observational data

Estimating treatment effect in randomized trial after control to treatment crossover using external controls

A Bayesian Quasi-Likelihood Design for Identifying the Minimum Effective Dose and Maximum Utility Dose in Dose-Ranging Studies

Validation of a Difference-in-Differences Investigation Tool (DiD IT) for quantifying local outbreaks

Statistical Power for Estimating Treatment Effects Using Difference-in-Differences and Comparative Interrupted Time Series Estimators With Variation in Treatment Timing

A Semiparametric Instrumented Difference-in-Differences Approach to Policy Learning

Causal Inference With Two Versions of Treatment

Regression discontinuity design studies: a guide for health researchers

Bayesian variable selection in hierarchical difference-in-differences models

Generalized Synthetic Control Method: Causal Inference with Interactive Fixed Effects Models