Abstract:When the distributional assumptions for a t-test are not met, the default position of many analysts is to resort to a rank-based test, such as the Wilcoxon-Mann-Whitney Test to compare the difference in means between two samples. The Wilcoxon-Mann-Whitney Test presents no danger of tied observations when the observations in the data are continuous. However, in practice, observations are discretized due various logical reasons, or the data are ordinal in nature. When ranks are tied, most textbooks recommend using mid-ranks to replace the tied ranks, a practice that affects the distribution of the Wilcoxon-Mann-Whitney Test under the null hypothesis. Other methods for breaking ties have also been proposed. In this study, we examine four tie-breaking methods-average-scores, mid-ranks, jittering, and omission-for their effects on Type I and Type II error of the Wilcoxon-Mann-Whitney Test and the two-sample t-test for various combinations of sample sizes, underlying population distributions, and percentages of tied observations. We use the results to determine the maximum percentage of ties for which the power and size are seriously affected, and for which method of tie-breaking results in the best Type I and Type II error properties. Not surprisingly, the underlying population distribution of the data has less of an effect on the Wilcoxon-Mann-Whitney Test than on the t-test. Surprisingly, we find that the jittering and omission methods tend to hold Type I error at the nominal level, even for small sample sizes, with no substantial sacrifice in terms of Type II error. Furthermore, the t-test and the Wilcoxon-Mann-Whitney Test are equally effected by ties in terms of Type I and Type II error; therefore, we recommend omitting tied observations when they occur for both the two-sample t-test and the Wilcoxon-Mann-Whitney due to the bias in Type I error that is created when tied observations are left in the data, in the case of the t-test, or adjusted using mid-ranks or average-scores, in the case of the Wilcoxon-Mann-Whitney.

The two-sample t test: pre-testing its assumptions does not pay off

The New and Improved Two-Sample t Test

Efficient three-stage $t$-tests

Case for omitting tied observations in the two-sample t-test and the Wilcoxon-Mann-Whitney Test

The effects of violations of assumptions underlying the t test.

t-tests, non-parametric tests, and large studies—a paradox of statistical practice?

Robustness of the T Test: A Guide for Researchers on Effect of Violations of Assumptions

ISSUES IN BIOMEDICAL STATISTICS: COMPARING MEANS UNDER NORMAL DISTRIBUTION THEORY

More about the basic assumptions of t-test: normality and sample size

Incorporation of Sparsity Information in Large-scale Multiple Two-sample $t$ Tests

Comparing type 1 and type 2 error rates of different tests for heterogeneous treatment effects

Testing in the Presence of Nuisance Parameters: Some Comments on Tests Post-Model-Selection and Random Critical Values

Optimal Sample Size Planning for the Wilcoxon-Mann-Whitney-Test

Wilcoxon-Mann-Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules

Are the tests overpowered or underpowered? A unified solution to correctly specify type I errors in design of clinical trials for two sample proportions

Testing for equivalence of pre-trends in Difference-in-Differences estimation

A noniterative sample size procedure for tests based on t distributions

Two-sample high dimensional mean test based on prepivots

Welch's t test is more sensitive to real world violations of distributional assumptions than student's t test but logistic regression is more robust than either

A note on the Wilcoxon-Mann-Whitney test and tied observations