Abstract:Causal effects are commonly defined as comparisons of the potential outcomes under treatment and control, but this definition is threatened by the possibility that either the treatment or the control condition is not well defined, existing instead in more than one version. This is often a real possibility in nonexperimental or observational studies of treatments because these treatments occur in the natural or social world without the laboratory control needed to ensure identically the same treatment or control condition occurs in every instance. We consider the simplest case: Either the treatment condition or the control condition exists in two versions that are easily recognized in the data but are of uncertain, perhaps doubtful, relevance, for example, branded Advil versus generic ibuprofen. Common practice does not address versions of treatment: Typically, the issue is either ignored or explicitly stated but assumed to be absent. Common practice is reluctant to address two versions of treatment because the obvious solution entails dividing the data into two parts with two analyses, thereby (a) reducing power to detect versions of treatment in each part, (b) creating problems of multiple inference in coordinating the two analyses, and (c) failing to report a single primary analysis that uses everyone. We propose and illustrate a new method of analysis that begins with a single primary analysis of everyone that would be correct if the two versions do not differ, adds a second analysis that would be correct were there two different effects for the two versions, controls the family-wise error rate in all assertions made by the several analyses, and yet pays no price in power to detect a constant treatment effect in the primary analysis of everyone. Our method can be applied to analyses of constant additive treatment effects on continuous outcomes. Unlike conventional simultaneous inferences, the new method is coordinating several analyses that are valid under different assumptions, so that one analysis would never be performed if one knew for certain that the assumptions of the other analysis are true. It is a multiple assumptions problem rather than a multiple hypotheses problem. We discuss the relative merits of the method with respect to more conventional approaches to analyzing multiple comparisons. The method is motivated and illustrated using a study of the possibility that repeated head trauma in high school football causes an increase in risk of early onset cognitive decline.

The Difference Between Causal Analysis and Predictive Models: Response to “Comment on Young and Holsteen (2017)”

Reply to "Comment on 'Experimentally adjudicating between different causal accounts of Bell-inequality violations via statistical model selection'"

Conflating marginal and conditional treatment effects: Comments on 'Assessing the performance of population adjustment methods for anchored indirect comparisons: A simulation study'

A comparison of methods for model selection when estimating individual treatment effects

A Comparative Study of Design-Based and Analysis-Based Approaches to Causal Inference with Observational Data

Causal Inference With Two Versions of Treatment

Causal definitions versus casual estimation: Reply to Valente et al. (2022).

For Objective Causal Inference, Design Trumps Analysis

On Associative Confounder Bias

Counterfactual Prediction Under Outcome Measurement Error

Propensity Score–based Methods Versus MTE-based Methods in Causal Inference

Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference

Rejoinder: Causal inference with misspecified exposure mappings: separating definitions and assumptions

Are causal effect estimations enough for optimal recommendations under multitreatment scenarios?

Uncertainty of Statistical Models and Propensity Score Methods

MODE: Minimax Optimal Deterministic Experiments for Causal Inference in the Presence of Covariates

A causal viewpoint on prediction model performance under changes in case-mix: discrimination and calibration respond differently for prognosis and diagnosis predictions

A Common Misassumption in Online Experiments with Machine Learning Models

Classifying Treatment Responders Under Causal Effect Monotonicity

Mixed Effects Models are Sometimes Terrible

Objective causal predictions from observational data