Abstract:While randomized trials may be the gold standard for evaluating the effectiveness of the treatment intervention, in some special circumstances, single-arm clinical trials utilizing external control may be considered. The causal treatment effect of interest for single-arm studies is usually the average treatment effect on the treated (ATT) rather than the average treatment effect (ATE). Although methods have been developed to estimate the ATT, the selection and use of these methods require a thorough comparison and in-depth understanding of the advantages and disadvantages of these methods. In this study, we conducted simulations under different identifiability assumptions to compare the performance metrics (e.g., bias, standard deviation (SD), mean squared error (MSE), type I error rate) for a variety of methods, including the regression model, propensity score matching, Mahalanobis distance matching, coarsened exact matching, inverse probability weighting, augmented inverse probability weighting (AIPW), AIPW with SuperLearner, and targeted maximum likelihood estimator (TMLE) with SuperLearner. Our simulation results demonstrate that the doubly robust methods in general have smaller biases than other methods. In terms of SD, nonmatching methods in general have smaller SDs than matching-based methods. The performance of MSE is a trade-off between the bias and SD, and no method consistently performs better in term of MSE. The identifiability assumptions are critical to the models' performance: violation of the positivity assumption can lead to a significant inflation of type I errors in some methods; violation of the unconfoundedness assumption can lead to a large bias for all methods... (Further details are available in the main body of the paper).

A computational approach to identification of treatment effects for policy evaluation

Estimating individual treatment effect: generalization bounds and algorithms

IDENTIFICATION AND INFERENCE FOR MARGINAL AVERAGE TREATMENT EFFECT ON THE TREATED WITH AN INSTRUMENTAL VARIABLE.

Selective Machine Learning of the Average Treatment Effect with an Invalid Instrumental Variable

Marginal Treatment Effects in the Absence of Instrumental Variables

ASSESSING THE TREATMENT EFFECT HETEROGENEITY WITH A LATENT VARIABLE

The Finite Sample Performance of Instrumental Variable-Based Estimators of the Local Average Treatment Effect When Controlling for Covariates

A Differential Effect Approach to Partial Identification of Treatment Effects

Identification and Estimation of Heterogeneous Treatment Effects under Non-compliance or Non-ignorable assignment

Estimation of the Local Conditional Tail Average Treatment Effect

Program Evaluation and Causal Inference with High-Dimensional Data

Doubly Robust Estimation of Local Average Treatment Effects Using Inverse Probability Weighted Regression Adjustment

Estimations of the Local Conditional Tail Average Treatment Effect

Dynamic Local Average Treatment Effects

Identification-robust inference for the LATE with high-dimensional covariates

Multiple Treatments with Strategic Interaction

Effect or Treatment Heterogeneity? Policy Evaluation with Aggregated and Disaggregated Treatments

The Informativeness of Combined Experimental and Observational Data under Dynamic Selection

Partial Identification of the Average Treatment Effect with Stochastic Counterfactuals and Discordant Twins

Policy Relevant Treatment Effects with Multidimensional Unobserved Heterogeneity

A comparison of methods for estimating the average treatment effect on the treated for externally controlled trials