Abstract:Evaluating the treatment effects has become an important topic for many applications. However, most existing literature focuses mainly on the average treatment effects. When the individual effects are heavy-tailed or have outlier values, not only may the average effect not be appropriate for summarizing the treatment effects, but also the conventional inference for it can be sensitive and possibly invalid due to poor large-sample approximations. In this paper we focus on quantiles of individual effects, which can be more robust measures of treatment effects in the presence of extreme individual effects. Moreover, our inference for quantiles of individual effects are purely randomization-based, which avoids any distributional assumption on the units. We first consider inference for stratified randomized experiments, extending the recent work of Caughey et al. (2021). The calculation of valid $p$-values for testing null hypotheses on quantiles of individual effects involves linear integer programming, which is generally NP hard. To overcome this issue, we propose a greedy algorithm with a certain optimal transformation, which has much lower computational cost, still leads to valid $p$-values and is less conservative than the usual relaxation by dropping the integer constraint. We then extend our approach to matched observational studies and propose sensitivity analysis to investigate to what extent our inference on quantiles of individual effects is robust to unmeasured confounding. Both the randomization inference and sensitivity analysis are simultaneously valid for all quantiles of individual effects, which are actually free lunches added to the conventional analysis assuming constant effects. Furthermore, the inference results can be easily visualized and interpreted.

Identification of quantile treatment effects in difference-in-differences settings with staggered adoption

A Generalized Difference-in-Differences Estimator for Randomized Stepped-Wedge and Observational Staggered Adoption Settings

Decomposing Triple-Differences Regression under Staggered Adoption

Simple Approaches to Nonlinear Difference-in-Differences with Panel Data

Efficient Estimation for Staggered Rollout Designs

Fused Extended Two-Way Fixed Effects for Difference-in-Differences With Staggered Adoptions

Difference-in-Differences Estimator of Quantile Treatment Effect on the Treated

Designing Difference-in-Difference Studies with Staggered Treatment Adoption: Key Concepts and Practical Guidelines

The chained difference-in-differences

A Differential Effect Approach to Partial Identification of Treatment Effects

Difference-in-Differences Meets Tree-based Methods: Heterogeneous Treatment Effects Estimation with Unmeasured Confounding.

Treatment Effect Quantiles in Stratified Randomized Experiments and Matched Observational Studies

Revisiting Event-Study Designs: Robust and Efficient Estimation

Uniformly Semiparametric Ecient Estimation of Treatment Eects with a Continuous Treatment

Understanding Difference-in-differences methods to evaluate policy effects with staggered adoption: an application to Medicaid and HIV

Empirical Likelihood Based Tests for Detecting the Presence of Significant Predictors in Marginal Quantile Regression

Uncertainty Quantification in Synthetic Controls with Staggered Treatment Adoption

Randomization Tests in Observational Studies with Staggered Adoption of Treatment

Estimation of Heterogeneous Treatment Effects Using Quantile Regression with Interactive Fixed Effects

Set Identification of Panel Data Models with Interactive Effects Via Quantile Restrictions

Sequential Synthetic Difference in Differences