Abstract:This paper studies inference in randomized controlled trials with covariate-adaptive randomization when there are multiple treatments. More specifically, we study in this setting inference about the average effect of one or more treatments relative to other treatments or a control. As in Bugni et al.(2018), covariate-adaptive randomization refers to randomization schemes that first stratify according to baseline covariates and then assign treatment status so as to achieve “balance” within each stratum. Importantly, in contrast to Bugni et al. (2018), we not only allow for multiple treatments, but further allow for the proportion of units being assigned to each of the treatments to vary across strata. We first study the properties of estimators derived from a “fully saturated” linear regression, i.e., a linear regression of the outcome on all interactions between indicators for each of the treatments and indicators for each of the strata. We show that tests based on these estimators using the usual heteroskedasticity-consistent estimator of the asymptotic variance are invalid in the sense that they may have limiting rejection probability under the null hypothesis strictly greater than the nominal level; on the other hand, tests based on these estimators and suitable estimators of the asymptotic variance that we provide are exact in the sense that they have limiting rejection probability under the null hypothesis equal to the nominal level. For the special case in which the target proportion of units being assigned to each of the treatments does not vary across strata, we additionally consider tests based on estimators derived from a linear regression with “strata fixed effects,” i.e., a linear regression of the outcome on indicators for each of the treatments and indicators for each of the strata. We show that tests based on these estimators using the usual heteroskedasticity-consistent estimator of the asymptotic variance are conservative in the sense that they have limiting rejection probability under the null hypothesis no greater than and typically strictly less than the nominal level, but tests based on these estimators and suitable estimators of the asymptotic variance that we provide are exact, thereby generalizing results in Bugni et al. (2018) for the case of a single treatment to multiple treatments. A simulation study and an empirical application illustrate the practical relevance of our theoretical results.

Inferring the Effect of a Confounded Treatment by Calibrating Resistant Population's Variance

Causal Inference for a Hidden Treatment

Estimation and Inference on Treatment Effects Under Treatment-Based Sampling

Accounting for Missing Covariates in Heterogeneous Treatment Estimation

Long-term Causal Inference under Persistent Confounding Via Data Combination

Causal inference for interfering units with cluster and population level treatment allocation programs

Identifying Effects of Multiple Treatments in the Presence of Unmeasured Confounding

A Differential Effect Approach to Partial Identification of Treatment Effects

Inference under Covariate-Adaptive Randomization with Multiple Treatments

Using Standard Tools from Finite Population Sampling to Improve Causal Inference for Complex Experiments

Causal Inference in Rebuilding and Extending the Recondite Bridge Between Finite Population Sampling and Experimental Design

Conformal Inference of Counterfactuals and Individual Treatment Effects

Causal Inference When Counterfactuals Depend on the Proportion of All Subjects Exposed

Treatment Effect Analysis for Pairs with Endogenous Treatment Takeup

Selective Machine Learning of the Average Treatment Effect with an Invalid Instrumental Variable

Inferring the Long-Term Causal Effects of Long-Term Treatments from Short-Term Experiments

Valid Causal Inference with (Some) Invalid Instruments

Efficient Covariate Balancing for the Local Average Treatment Effect

Generalization Bounds for Estimating Causal Effects of Continuous Treatments

Inference under covariate-adaptive randomization with imperfect compliance

Causal inference under transportability assumptions for conditional relative effect measures