Abstract:Evaluating the treatment effects has become an important topic for many applications. However, most existing literature focuses mainly on the average treatment effects. When the individual effects are heavy-tailed or have outlier values, not only may the average effect not be appropriate for summarizing the treatment effects, but also the conventional inference for it can be sensitive and possibly invalid due to poor large-sample approximations. In this paper we focus on quantiles of individual effects, which can be more robust measures of treatment effects in the presence of extreme individual effects. Moreover, our inference for quantiles of individual effects are purely randomization-based, which avoids any distributional assumption on the units. We first consider inference for stratified randomized experiments, extending the recent work of Caughey et al. (2021). The calculation of valid $p$-values for testing null hypotheses on quantiles of individual effects involves linear integer programming, which is generally NP hard. To overcome this issue, we propose a greedy algorithm with a certain optimal transformation, which has much lower computational cost, still leads to valid $p$-values and is less conservative than the usual relaxation by dropping the integer constraint. We then extend our approach to matched observational studies and propose sensitivity analysis to investigate to what extent our inference on quantiles of individual effects is robust to unmeasured confounding. Both the randomization inference and sensitivity analysis are simultaneously valid for all quantiles of individual effects, which are actually free lunches added to the conventional analysis assuming constant effects. Furthermore, the inference results can be easily visualized and interpreted.

Quantile‐Based Test for Heterogeneous Treatment Effects

A Nonparametric Test Of Heterogeneity In Conditional Quantile Treatment Effects

Multivariate quantile-based permutation tests with application to functional data

Estimation of Heterogeneous Treatment Effects Using Quantile Regression with Interactive Fixed Effects

Testing for Quantile Sample Selection

Empirical Likelihood Based Tests for Detecting the Presence of Significant Predictors in Marginal Quantile Regression

Generalized Quantile Treatment Effect: A Flexible Bayesian Approach Using Quantile Ratio Smoothing

Treatment Effect Quantiles in Stratified Randomized Experiments and Matched Observational Studies

Global quantile regression

Nonparametric Tests for Treatment Effect Heterogeneity with Duration Outcomes

On Quantile Treatment Effects, Rank Similarity, and Variation of Instrumental Variables

Permutation Tests at Nonparametric Rates

TESTING FOR UNOBSERVED HETEROGENEOUS TREATMENT EFFECTS WITH OBSERVATIONAL DATA

Difference-in-Differences Estimator of Quantile Treatment Effect on the Treated

The Classification Permutation Test: A Nonparametric Test for Equality of Multivariate Distributions

Multiply Robust Estimation of Quantile Treatment Effects with Missing Responses

A Nonparametric Test of Heterogeneous Treatment Effects under Interference

Quantile-Quantile Methodology -- Detailed Results

Testing Structural Change in Conditional Distributions via Quantile Regressions

Quantile ratio regression

Reprint: Hypothesis testing on high dimensional quantile regression