Abstract:When the distributional assumptions for a t-test are not met, the default position of many analysts is to resort to a rank-based test, such as the Wilcoxon-Mann-Whitney Test to compare the difference in means between two samples. The Wilcoxon-Mann-Whitney Test presents no danger of tied observations when the observations in the data are continuous. However, in practice, observations are discretized due various logical reasons, or the data are ordinal in nature. When ranks are tied, most textbooks recommend using mid-ranks to replace the tied ranks, a practice that affects the distribution of the Wilcoxon-Mann-Whitney Test under the null hypothesis. Other methods for breaking ties have also been proposed. In this study, we examine four tie-breaking methods-average-scores, mid-ranks, jittering, and omission-for their effects on Type I and Type II error of the Wilcoxon-Mann-Whitney Test and the two-sample t-test for various combinations of sample sizes, underlying population distributions, and percentages of tied observations. We use the results to determine the maximum percentage of ties for which the power and size are seriously affected, and for which method of tie-breaking results in the best Type I and Type II error properties. Not surprisingly, the underlying population distribution of the data has less of an effect on the Wilcoxon-Mann-Whitney Test than on the t-test. Surprisingly, we find that the jittering and omission methods tend to hold Type I error at the nominal level, even for small sample sizes, with no substantial sacrifice in terms of Type II error. Furthermore, the t-test and the Wilcoxon-Mann-Whitney Test are equally effected by ties in terms of Type I and Type II error; therefore, we recommend omitting tied observations when they occur for both the two-sample t-test and the Wilcoxon-Mann-Whitney due to the bias in Type I error that is created when tied observations are left in the data, in the case of the t-test, or adjusted using mid-ranks or average-scores, in the case of the Wilcoxon-Mann-Whitney.

An Inverse Normal Transformation Solution for the comparison of two samples that contain both paired observations and independent observations

The New and Improved Two-Sample t Test

Mistaken identities lead to missed opportunities: Testing for mean differences in partially matched data

A Comparative Review of Methods for Comparing Means Using Partially Paired Data

A noniterative sample size procedure for tests based on t distributions

ISSUES IN BIOMEDICAL STATISTICS: COMPARING MEANS UNDER NORMAL DISTRIBUTION THEORY

A robust test approach for equality of mean vectors of two independent groups under the multivariate Behrens-Fisher problem

Case for omitting tied observations in the two-sample t-test and the Wilcoxon-Mann-Whitney Test

Two-sample testing in non-sparse high-dimensional linear models

Optimal Weighted Two-Sample T-Test with Partially Paired Data in a Unified Framework

On Testing the Mean Equivalence of Treatments from Correlated Normal Populations

On Steel's Test with Ties

A More Powerful Two-Sample Test in High Dimensions using Random Projection

Te Test: A New Non-asymptotic T-test for Behrens-Fisher Problems

Generalized kernel two-sample tests

Bayesian Optimal Two-sample Tests in High-dimension

Inference on order restricted means of inverse Gaussian populations under heteroscedasticity

An Improvement on the Hotelling $T^2$ Test Using the Ledoit-Wolf Nonlinear Shrinkage Estimator

The AUGUST Two-Sample Test: Powerful, Interpretable, and Fast

Large-Scale Two-Sample Comparison of Support Sets

The two-sample t test: pre-testing its assumptions does not pay off