Abstract:BackgroundThe rank correlation test introduced by Begg and Mazumdar is extensively used in meta-analysis to test for publication bias in clinical and epidemiological studies. It is based on correlating the standardized treatment effect with the variance of the treatment effect using Kendall’s tau as the measure of association. To our knowledge, the operational characteristics regarding the significance level of the test have not, however, been fully assessed.MethodsWe propose an alternative rank correlation test to improve the error rates of the original Begg and Mazumdar test. This test is based on the simulated distribution of the estimated measure of association, conditional on sampling variances. Furthermore, Spearman’s rho is suggested as an alternative rank correlation coefficient. The attained level and power of the tests are studied by simulations of meta-analyses assuming the fixed effects model.ResultsThe significance levels of the original Begg and Mazumdar test often deviate considerably from the nominal level, the null hypothesis being rejected too infrequently. It is proven mathematically that the assumptions for using the rank correlation test are not strictly satisfied. The pairs of variables fail to be independent, and there is a correlation between the standardized effect sizes and sampling variances under the null hypothesis of no publication bias. In the meta-analysis setting, the adverse consequences of a false negative test are more profound than the disadvantages of a false positive test. Our alternative test improves the error rates in fixed effects meta-analysis. Its significance level equals the nominal value, and the Type II error rate is reduced. In small data sets Spearman’s rho should be preferred to Kendall’s tau as the measure of association.ConclusionsAs the attained significance levels of the test introduced by Begg and Mazumdar often deviate greatly from the nominal level, modified rank correlation tests, improving the error rates, should be preferred when testing for publication bias assuming fixed effects meta-analysis.

The effect of screening for publication bias on the outcomes of meta‐analyses

Empirical Comparison of Publication Bias Tests in Meta-Analysis

Sensitivity analysis for publication bias in meta‐analyses

The power functions of Begg's and Egger's tests for publication bias

Systematic Evaluation and Comparison of Statistical Tests for Publication Bias

Using selection models to assess sensitivity to publication bias: A tutorial and call for more routine use

Hybrid test for publication bias in meta-analysis

Examining publication bias—a simulation-based evaluation of statistical tests on publication bias

Footprint of publication selection bias on meta‐analyses in medicine, environmental sciences, psychology, and economics

Improving the error rates of the Begg and Mazumdar test for publication bias in fixed effects meta-analysis

Methods for testing publication bias in ecological and evolutionary meta‐analyses

A modified regression method to test publication bias in meta-analyses with binary outcomes

Estimating the change in meta-analytic effect size estimates after the application of publication bias adjustment methods.

Investigation of publication bias in meta-analyses of diagnostic test accuracy: a meta-epidemiological study

Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis

Publication Bias in the Organizational Sciences

Testing for Publication Bias in Diagnostic Meta-Analysis: A Simulation Study

A regression-based method for detecting publication bias in multivariate meta-analysis

Testing for publication bias in meta-analysis under Copas selection model

Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods

A likelihood based sensitivity analysis for publication bias on summary ROC in meta-analysis of diagnostic test accuracy