Abstract:While randomized trials may be the gold standard for evaluating the effectiveness of the treatment intervention, in some special circumstances, single-arm clinical trials utilizing external control may be considered. The causal treatment effect of interest for single-arm studies is usually the average treatment effect on the treated (ATT) rather than the average treatment effect (ATE). Although methods have been developed to estimate the ATT, the selection and use of these methods require a thorough comparison and in-depth understanding of the advantages and disadvantages of these methods. In this study, we conducted simulations under different identifiability assumptions to compare the performance metrics (e.g., bias, standard deviation (SD), mean squared error (MSE), type I error rate) for a variety of methods, including the regression model, propensity score matching, Mahalanobis distance matching, coarsened exact matching, inverse probability weighting, augmented inverse probability weighting (AIPW), AIPW with SuperLearner, and targeted maximum likelihood estimator (TMLE) with SuperLearner. Our simulation results demonstrate that the doubly robust methods in general have smaller biases than other methods. In terms of SD, nonmatching methods in general have smaller SDs than matching-based methods. The performance of MSE is a trade-off between the bias and SD, and no method consistently performs better in term of MSE. The identifiability assumptions are critical to the models' performance: violation of the positivity assumption can lead to a significant inflation of type I errors in some methods; violation of the unconfoundedness assumption can lead to a large bias for all methods... (Further details are available in the main body of the paper).

Cross-distribution Comparison of Five linear Equating Methods Based on Mixed Model of Item Response Theory

Using a Linear Regression Method to Detect Outliers in IRT Common Item Equating

Harmonizing the CBCL and SDQ ADHD scores by using linear equating, kernel equating, item response theory and machine learning methods

Applying Unidimensional and Multidimensional Item Response Theory Models in Testlet-Based Reading Assessment

2PL Model: Compare Generalized Linear Mixed Model with Latent Variable Model based IRT framework

On Fitting Generalized Linear Mixed-Effects Models for Binary Responses Using Different Statistical Packages.

Comparison of confirmatory factor analysis estimation methods on mixed-format data

Item-Weighted Likelihood Method for Measuring Growth in Longitudinal Study With Tests Composed of Both Dichotomous and Polytomous Items

New Robust Scale Transformation Methods in the Presence of Outlying Common Items.

Maximum Marginal Likelihood Estimation of a Monotonic Polynomial Generalized Partial Credit Model with Applications to Multiple Group Analysis

Leveraging LLM-Respondents for Item Evaluation: a Psychometric Analysis

Comparison of Outlier Detection Methods in NEAT Design

Doubly robust estimation of average treatment effect revisited

Assessing method agreement for paired repeated binary measurements administered by multiple raters

A comparative study on the performance of maximum likelihood, generalized least square, scale-free least square, partial least square and consistent partial least square estimators in structural equation modeling

Subgroup Analysis of Linear Models with Measurement Error

A Comparison of Differential Item Functioning Detection Methods in Cognitive Diagnostic Models

Testing the equality of response rate functions for paired binary data with multiple groups

Efficient and powerful equivalency test on combined mean and variance with application to diagnostic device comparison studies

A comparison of methods for estimating the average treatment effect on the treated for externally controlled trials

MIMIC approach to assessing differential item functioning with control of extreme response style