Abstract:This paper studies inference in randomized controlled trials with covariate-adaptive randomization when there are multiple treatments. More specifically, we study in this setting inference about the average effect of one or more treatments relative to other treatments or a control. As in Bugni et al.(2018), covariate-adaptive randomization refers to randomization schemes that first stratify according to baseline covariates and then assign treatment status so as to achieve “balance” within each stratum. Importantly, in contrast to Bugni et al. (2018), we not only allow for multiple treatments, but further allow for the proportion of units being assigned to each of the treatments to vary across strata. We first study the properties of estimators derived from a “fully saturated” linear regression, i.e., a linear regression of the outcome on all interactions between indicators for each of the treatments and indicators for each of the strata. We show that tests based on these estimators using the usual heteroskedasticity-consistent estimator of the asymptotic variance are invalid in the sense that they may have limiting rejection probability under the null hypothesis strictly greater than the nominal level; on the other hand, tests based on these estimators and suitable estimators of the asymptotic variance that we provide are exact in the sense that they have limiting rejection probability under the null hypothesis equal to the nominal level. For the special case in which the target proportion of units being assigned to each of the treatments does not vary across strata, we additionally consider tests based on estimators derived from a linear regression with “strata fixed effects,” i.e., a linear regression of the outcome on indicators for each of the treatments and indicators for each of the strata. We show that tests based on these estimators using the usual heteroskedasticity-consistent estimator of the asymptotic variance are conservative in the sense that they have limiting rejection probability under the null hypothesis no greater than and typically strictly less than the nominal level, but tests based on these estimators and suitable estimators of the asymptotic variance that we provide are exact, thereby generalizing results in Bugni et al. (2018) for the case of a single treatment to multiple treatments. A simulation study and an empirical application illustrate the practical relevance of our theoretical results.

Generalizing causal inferences from individuals in randomized trials to all trial-eligible individuals

Extending inferences from a randomized trial to a new target population

Generalizing and transporting causal inferences from randomized trials in the presence of trial engagement effects

Study designs for extending causal inferences from a randomized trial to a target population

Causal Inference for Comprehensive Cohort Studies

Regression-based estimation of heterogeneous treatment effects when extending inferences from a randomized trial to a target population

Prediction-powered Generalization of Causal Inferences

Generalizing treatment effects with incomplete covariates: Identifying assumptions and multiple imputation algorithms

Towards Generalizing Inferences from Trials to Target Populations

Extending inferences from a cluster randomized trial to a target population

Estimating subgroup effects in generalizability and transportability analyses

Sensitivity analysis using bias functions for studies extending inferences from a randomized trial to a target population

Center-specific causal inference with multicenter trials: reinterpreting trial evidence in the context of each participating center

Estimating individual treatment effect: generalization bounds and algorithms

Generalizing and transporting inferences about the effects of treatment assignment subject to non-adherence

Inference under Covariate-Adaptive Randomization with Multiple Treatments

Improving transportability of randomized controlled trial inference using robust prediction methods

Estimating the Causal Effects in Randomized Trials for Survival Data with A Cure Fraction and Non Compliance

Causal inference methods for combining randomized trials and observational studies: a review

Causal Inference for a Hidden Treatment

Matching Algorithms for Causal Inference with Multiple Treatments