Abstract:BackgroundThe rank correlation test introduced by Begg and Mazumdar is extensively used in meta-analysis to test for publication bias in clinical and epidemiological studies. It is based on correlating the standardized treatment effect with the variance of the treatment effect using Kendall’s tau as the measure of association. To our knowledge, the operational characteristics regarding the significance level of the test have not, however, been fully assessed.MethodsWe propose an alternative rank correlation test to improve the error rates of the original Begg and Mazumdar test. This test is based on the simulated distribution of the estimated measure of association, conditional on sampling variances. Furthermore, Spearman’s rho is suggested as an alternative rank correlation coefficient. The attained level and power of the tests are studied by simulations of meta-analyses assuming the fixed effects model.ResultsThe significance levels of the original Begg and Mazumdar test often deviate considerably from the nominal level, the null hypothesis being rejected too infrequently. It is proven mathematically that the assumptions for using the rank correlation test are not strictly satisfied. The pairs of variables fail to be independent, and there is a correlation between the standardized effect sizes and sampling variances under the null hypothesis of no publication bias. In the meta-analysis setting, the adverse consequences of a false negative test are more profound than the disadvantages of a false positive test. Our alternative test improves the error rates in fixed effects meta-analysis. Its significance level equals the nominal value, and the Type II error rate is reduced. In small data sets Spearman’s rho should be preferred to Kendall’s tau as the measure of association.ConclusionsAs the attained significance levels of the test introduced by Begg and Mazumdar often deviate greatly from the nominal level, modified rank correlation tests, improving the error rates, should be preferred when testing for publication bias assuming fixed effects meta-analysis.

Testing for Publication Bias in Diagnostic Meta-Analysis: A Simulation Study

Examining publication bias—a simulation-based evaluation of statistical tests on publication bias

Empirical Comparison of Publication Bias Tests in Meta-Analysis

A modified regression method to test publication bias in meta-analyses with binary outcomes

Hybrid test for publication bias in meta-analysis

Investigation of publication bias in meta-analyses of diagnostic test accuracy: a meta-epidemiological study

Systematic Evaluation and Comparison of Statistical Tests for Publication Bias

Improving the error rates of the Begg and Mazumdar test for publication bias in fixed effects meta-analysis

Copas-like selection model to correct publication bias in systematic review of diagnostic test studies

Operating Characteristics of a Rank Correlation Test for Publication Bias

Statistical inference for diagnostic test accuracy studies with multiple comparisons

A regression-based method for detecting publication bias in multivariate meta-analysis

Bias in meta-analysis detected by a simple, graphical test

Meta‐Analysis of Diagnostic Accuracy Studies With Multiple Thresholds: Comparison of Approaches in a Simulation Study

Simulation models for aggregated data meta-analysis: Evaluation of pooling effect sizes and publication biases

A likelihood based sensitivity analysis for publication bias on summary ROC in meta-analysis of diagnostic test accuracy

Using selection models to assess sensitivity to publication bias: A tutorial and call for more routine use

A likelihood-based sensitivity analysis for addressing publication bias in meta-analysis of diagnostic studies using exact likelihood

Funnel plots may show asymmetry in the absence of publication bias with continuous outcomes dependent on baseline risk: presentation of a new publication bias test

Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods

LFK index does not reliably detect small‐study effects in meta‐analysis: A simulation study