Abstract:BackgroundThe rank correlation test introduced by Begg and Mazumdar is extensively used in meta-analysis to test for publication bias in clinical and epidemiological studies. It is based on correlating the standardized treatment effect with the variance of the treatment effect using Kendall’s tau as the measure of association. To our knowledge, the operational characteristics regarding the significance level of the test have not, however, been fully assessed.MethodsWe propose an alternative rank correlation test to improve the error rates of the original Begg and Mazumdar test. This test is based on the simulated distribution of the estimated measure of association, conditional on sampling variances. Furthermore, Spearman’s rho is suggested as an alternative rank correlation coefficient. The attained level and power of the tests are studied by simulations of meta-analyses assuming the fixed effects model.ResultsThe significance levels of the original Begg and Mazumdar test often deviate considerably from the nominal level, the null hypothesis being rejected too infrequently. It is proven mathematically that the assumptions for using the rank correlation test are not strictly satisfied. The pairs of variables fail to be independent, and there is a correlation between the standardized effect sizes and sampling variances under the null hypothesis of no publication bias. In the meta-analysis setting, the adverse consequences of a false negative test are more profound than the disadvantages of a false positive test. Our alternative test improves the error rates in fixed effects meta-analysis. Its significance level equals the nominal value, and the Type II error rate is reduced. In small data sets Spearman’s rho should be preferred to Kendall’s tau as the measure of association.ConclusionsAs the attained significance levels of the test introduced by Begg and Mazumdar often deviate greatly from the nominal level, modified rank correlation tests, improving the error rates, should be preferred when testing for publication bias assuming fixed effects meta-analysis.

The power functions of Begg's and Egger's tests for publication bias

Systematic Evaluation and Comparison of Statistical Tests for Publication Bias

Empirical Comparison of Publication Bias Tests in Meta-Analysis

Examining publication bias—a simulation-based evaluation of statistical tests on publication bias

Improving the error rates of the Begg and Mazumdar test for publication bias in fixed effects meta-analysis

Methods for testing publication bias in ecological and evolutionary meta‐analyses

Hybrid test for publication bias in meta-analysis

Sensitivity analysis for publication bias in meta‐analyses

Operating Characteristics of a Rank Correlation Test for Publication Bias

A modified regression method to test publication bias in meta-analyses with binary outcomes

Effect Comparison Between Egger’s Test and Begg’s Test in Publication Bias Diagnosis in Meta-Analyses: Evidence from a Pilot Survey

A regression-based method for detecting publication bias in multivariate meta-analysis

Testing and adjusting for publication bias

The effect of screening for publication bias on the outcomes of meta‐analyses

P Value–driven Methods Were Underpowered to Detect Publication Bias: Analysis of Cochrane Review Meta-Analyses

Using selection models to assess sensitivity to publication bias: A tutorial and call for more routine use

Testing for Publication Bias in Diagnostic Meta-Analysis: A Simulation Study

Investigating and dealing with publication bias and other reporting biases in meta‐analyses of health research: A review

Estimating the change in meta-analytic effect size estimates after the application of publication bias adjustment methods.

Publication Bias in the Organizational Sciences

Footprint of publication selection bias on meta‐analyses in medicine, environmental sciences, psychology, and economics