Abstract:Evaluating the treatment effects has become an important topic for many applications. However, most existing literature focuses mainly on the average treatment effects. When the individual effects are heavy-tailed or have outlier values, not only may the average effect not be appropriate for summarizing the treatment effects, but also the conventional inference for it can be sensitive and possibly invalid due to poor large-sample approximations. In this paper we focus on quantiles of individual effects, which can be more robust measures of treatment effects in the presence of extreme individual effects. Moreover, our inference for quantiles of individual effects are purely randomization-based, which avoids any distributional assumption on the units. We first consider inference for stratified randomized experiments, extending the recent work of Caughey et al. (2021). The calculation of valid $p$-values for testing null hypotheses on quantiles of individual effects involves linear integer programming, which is generally NP hard. To overcome this issue, we propose a greedy algorithm with a certain optimal transformation, which has much lower computational cost, still leads to valid $p$-values and is less conservative than the usual relaxation by dropping the integer constraint. We then extend our approach to matched observational studies and propose sensitivity analysis to investigate to what extent our inference on quantiles of individual effects is robust to unmeasured confounding. Both the randomization inference and sensitivity analysis are simultaneously valid for all quantiles of individual effects, which are actually free lunches added to the conventional analysis assuming constant effects. Furthermore, the inference results can be easily visualized and interpreted.

Regression-adjusted Average Treatment Effect Estimates in Stratified Randomized Experiments

Regression-based multiple treatment effect estimation under covariate-adaptive randomization

Regression Analysis for Covariate-Adaptive Randomization: A Robust and Efficient Inference Perspective

Lasso-adjusted treatment effect estimation under covariate-adaptive randomization

Inference on Average Treatment Effect under Minimization and Other Covariate-Adaptive Randomization Methods

Randomization-based joint central limit theorem and efficient covariate adjustment in stratified $2^K$ factorial experiments

Rerandomization in Stratified Randomized Experiments

A Unified Framework for Covariate Adjustment under Stratified Randomization

A general theory of regression adjustment for covariate-adaptive randomization: OLS, Lasso, and beyond

Covariate Adjustment in Stratified Experiments

Treatment effect estimation under covariate-adaptive randomization with heavy-tailed outcomes

A Unified Framework for Covariate Adjustment Under Stratified Randomisation

Bootstrapping Inference of Average Treatment Effect in Completely Randomized Experiments with High-Dimensional Covariates

Blocking, rerandomization, and regression adjustment in randomized experiments with high-dimensional covariates

Treatment Effect Quantiles in Stratified Randomized Experiments and Matched Observational Studies

Sharp variance estimator and causal bootstrap in stratified randomized experiments

Tyranny-of-the-Minority Regression Adjustment in Randomized Experiments

Penalized Regression Adjusted Causal Effect Estimates in High Dimensional Randomized Experiments

Estimation and inference of average treatment effects under heterogeneous additive treatment effect model

Finely Stratified Rerandomization Designs

Regression Adjustment for Estimating Distributional Treatment Effects in Randomized Controlled Trials