Abstract:This paper studies inference in randomized controlled trials with covariate-adaptive randomization when there are multiple treatments. More specifically, we study in this setting inference about the average effect of one or more treatments relative to other treatments or a control. As in Bugni et al.(2018), covariate-adaptive randomization refers to randomization schemes that first stratify according to baseline covariates and then assign treatment status so as to achieve “balance” within each stratum. Importantly, in contrast to Bugni et al. (2018), we not only allow for multiple treatments, but further allow for the proportion of units being assigned to each of the treatments to vary across strata. We first study the properties of estimators derived from a “fully saturated” linear regression, i.e., a linear regression of the outcome on all interactions between indicators for each of the treatments and indicators for each of the strata. We show that tests based on these estimators using the usual heteroskedasticity-consistent estimator of the asymptotic variance are invalid in the sense that they may have limiting rejection probability under the null hypothesis strictly greater than the nominal level; on the other hand, tests based on these estimators and suitable estimators of the asymptotic variance that we provide are exact in the sense that they have limiting rejection probability under the null hypothesis equal to the nominal level. For the special case in which the target proportion of units being assigned to each of the treatments does not vary across strata, we additionally consider tests based on estimators derived from a linear regression with “strata fixed effects,” i.e., a linear regression of the outcome on indicators for each of the treatments and indicators for each of the strata. We show that tests based on these estimators using the usual heteroskedasticity-consistent estimator of the asymptotic variance are conservative in the sense that they have limiting rejection probability under the null hypothesis no greater than and typically strictly less than the nominal level, but tests based on these estimators and suitable estimators of the asymptotic variance that we provide are exact, thereby generalizing results in Bugni et al. (2018) for the case of a single treatment to multiple treatments. A simulation study and an empirical application illustrate the practical relevance of our theoretical results.

Randomization inference for treatment effect variation

Estimating treatment effect heterogeneity in randomized program evaluation

Inference under Covariate-Adaptive Randomization with Multiple Treatments

Estimating individual treatment effect: generalization bounds and algorithms

Asymptotic Theory of Rerandomization in Treatment-Control Experiments

Randomization Inference of Heterogeneous Treatment Effects under Network Interference

Selective Randomization Inference for Adaptive Experiments

Bridging the Gap Between Design and Analysis: Randomization Inference and Sensitivity Analysis for Matched Observational Studies with Treatment Doses

Randomization Tests in Observational Studies with Staggered Adoption of Treatment

To adjust or not to adjust? Estimating the average treatment effect in randomized experiments with missing covariates

Imputation-based Randomization Tests for Randomized Experiments with Interference

Treatment Effect Quantiles in Stratified Randomized Experiments and Matched Observational Studies

Berry-Esseen bounds for design-based causal inference with possibly diverging treatment levels and varying group sizes

What can the millions of random treatments in nonexperimental data reveal about causes?

Enhanced inference for distributions and quantiles of individual treatment effects in various experiments

Randomization Bias in Field Trials to Evaluate Targeting Methods

Inference for Two-stage Experiments under Covariate-Adaptive Randomization

Randomization inference with general interference and censoring

Randomization Tests for Adaptively Collected Data

Random-effect Based Test for Multinomial Logistic Regression: Choice of the Reference Level and Its Impact on the Testing

Analyzing two-stage experiments in the presence of interference