Software for Publication Bias

Michael Borenstein

DOI: https://doi.org/10.1002/0470870168.ch11

2006-03-30

Abstract:This chapter contains sections titled: Introduction Statistical Procedures Illustrative Example The Software Where Do We Go from Here? Comparing the Programs Acknowledgements References Introduction Statistical Procedures Illustrative Example The Software Where Do We Go from Here? Comparing the Programs Acknowledgements References

What problem does this paper attempt to address?

Examining publication bias—a simulation-based evaluation of statistical tests on publication bias

Andreas Schneck

DOI: https://doi.org/10.7717/peerj.4115

IF: 3.061

2017-11-30

PeerJ

Abstract:Background Publication bias is a form of scientific misconduct. It threatens the validity of research results and the credibility of science. Although several tests on publication bias exist, no in-depth evaluations are available that examine which test performs best for different research settings. Methods Four tests on publication bias, Egger’s test (FAT), p-uniform, the test of excess significance (TES), as well as the caliper test, were evaluated in a Monte Carlo simulation. Two different types of publication bias and its degree (0%, 50%, 100%) were simulated. The type of publication bias was defined either as file-drawer , meaning the repeated analysis of new datasets, or p-hacking , meaning the inclusion of covariates in order to obtain a significant result. In addition, the underlying effect ( β = 0, 0.5, 1, 1.5), effect heterogeneity, the number of observations in the simulated primary studies ( N = 100, 500), and the number of observations for the publication bias tests ( K = 100, 1,000) were varied. Results All tests evaluated were able to identify publication bias both in the file-drawer and p-hacking condition. The false positive rates were, with the exception of the 15%- and 20%-caliper test, unbiased. The FAT had the largest statistical power in the file-drawer conditions, whereas under p-hacking the TES was, except under effect heterogeneity, slightly better. The CTs were, however, inferior to the other tests under effect homogeneity and had a decent statistical power only in conditions with 1,000 primary studies. Discussion The FAT is recommended as a test for publication bias in standard meta-analyses with no or only small effect heterogeneity. If two-sided publication bias is suspected as well as under p-hacking the TES is the first alternative to the FAT. The 5%-caliper test is recommended under conditions of effect heterogeneity and a large number of primary studies, which may be found if publication bias is examined in a discipline-wide setting when primary studies cover different research problems.

multidisciplinary sciences
Publication Bias in the Organizational Sciences

Sven Kepes,George C. Banks,Michael McDaniel,Deborah L. Whetzel

DOI: https://doi.org/10.1177/1094428112452760

IF: 8.247

2012-07-31

Organizational Research Methods

Abstract:Publication bias poses multiple threats to the accuracy of meta-analytically derived effect sizes and related statistics. Unfortunately, a review of the literature indicates that unlike meta-analytic reviews in medicine, research in the organizational sciences tends to pay little attention to this issue. In this article, the authors introduce advances in meta-analytic techniques from the medical and related sciences for a comprehensive assessment and evaluation of publication bias. The authors illustrate their use on a data set on employment interview validities. Using multiple methods, including contour-enhanced funnel plots, trim and fill, Egger’s test of the intercept, Begg and Mazumdar’s rank correlation, meta-regression, cumulative meta-analysis, and selection models, the authors find limited evidence of publication bias in the studied data.

management,psychology, applied
The power functions of Begg's and Egger's tests for publication bias

Haben Michael

DOI: https://doi.org/10.1111/stan.12364

2024-11-09

Statistica Neerlandica

Abstract:Publication bias undermines the results of meta‐analyses and other systematic reviews. Applying a formal test for publication bias is therefore a part of many protocols for conducting a meta‐analysis. To be effective in this role, the tests must have good power to detect the presence of publication bias. We derive the local limiting power functions and study the Pitman relative efficiency for two popular tests, Begg's and Egger's tests. The analysis suggests that the differences between the two arise out of built‐in robustness properties of Begg's test. These work to its detriment in the commonly used Gaussian model, but leave it with better efficiency when study effects are heavy‐tailed or skew.

statistics & probability
Systematic Evaluation and Comparison of Statistical Tests for Publication Bias

Yasuaki Hayashino,Yoshinori Noguchi,Tsuguya Fukui

DOI: https://doi.org/10.2188/jea.15.235

2005-01-01

Journal of Epidemiology

Abstract:BACKGROUND: This study evaluates the statistical and discriminatory powers of three statistical test methods (Begg's, Egger's, and Macaskill's) to detect publication bias in meta-analyses.METHODS: The data sources were 130 reviews from the Cochrane Database of Systematic Reviews 2002 issue, which considered a binary endpoint and contained 10 or more individual studies. Funnel plots with observers'agreements were selected as a reference standard. We evaluated a trade-off between sensitivity and specificity by varying cut-off p-values, power of statistical tests given fixed false positive rates, and area under the receiver operating characteristic curve.RESULTS: In 36 reviews, 733 original studies evaluated 2,874,006 subjects. The number of trials included in each ranged from 10 to 70 (median 14.5). Given that the false positive rate was 0.1, the sensitivity of Egger's method was 0.93, and was larger than that of Begg's method (0.86) and Macaskill's method (0.43). The sensitivities of three statistical tests increased as the cut-off p-values increased without a substantial decrement of specificities. The area under the ROC curve of Egger's method was 0.955 (95% confidence interval, 0.889-1.000) and was not different from that of Begg's method (area=0.913, p=0.2302), but it was larger than that of Macaskill's method (area=0.719, p=0.0116).CONCLUSION: Egger's linear regression method and Begg's method had stronger statistical and discriminatory powers than Macaskill's method for detecting publication bias given the same type I error level. The power of these methods could be improved by increasing the cut-off p-value without a substantial increment of false positive rate.

public, environmental & occupational health
Empirical Comparison of Publication Bias Tests in Meta-Analysis

Lifeng Lin,Haitao Chu,Mohammad Hassan Murad,Chuan Hong,Zhiyong Qu,Stephen R. Cole,Yong Chen

DOI: https://doi.org/10.1007/s11606-018-4425-7

IF: 5.7

2018-04-16

Journal of General Internal Medicine

Abstract:ABSTRACTBackgroundDecision makers rely on meta-analytic estimates to trade off benefits and harms. Publication bias impairs the validity and generalizability of such estimates. The performance of various statistical tests for publication bias has been largely compared using simulation studies and has not been systematically evaluated in empirical data.MethodsThis study compares seven commonly used publication bias tests (i.e., Begg’s rank test, trim-and-fill, Egger’s, Tang’s, Macaskill’s, Deeks’, and Peters’ regression tests) based on 28,655 meta-analyses available in the Cochrane Library.ResultsEgger’s regression test detected publication bias more frequently than other tests (15.7% in meta-analyses of binary outcomes and 13.5% in meta-analyses of non-binary outcomes). The proportion of statistically significant publication bias tests was greater for larger meta-analyses, especially for Begg’s rank test and the trim-and-fill method. The agreement among Tang’s, Macaskill’s, Deeks’, and Peters’ regression tests for binary outcomes was moderately strong (most κ’s were around 0.6). Tang’s and Deeks’ tests had fairly similar performance (κ > 0.9). The agreement among Begg’s rank test, the trim-and-fill method, and Egger’s regression test was weak or moderate (κ < 0.5).ConclusionsGiven the relatively low agreement between many publication bias tests, meta-analysts should not rely on a single test and may apply multiple tests with various assumptions. Non-statistical approaches to evaluating publication bias (e.g., searching clinical trials registries, records of drug approving agencies, and scientific conference proceedings) remain essential.

medicine, general & internal,health care sciences & services
Adjusting for Publication Bias in JASP and R: Selection Models, PET-PEESE, and Robust Bayesian Meta-Analysis

František Bartoš,Maximilian Maier,Daniel S. Quintana,Eric-Jan Wagenmakers

DOI: https://doi.org/10.1177/25152459221109259

2022-07-01

Advances in Methods and Practices in Psychological Science

Abstract:Meta-analyses are essential for cumulative science, but their validity can be compromised by publication bias. To mitigate the impact of publication bias, one may apply publication-bias-adjustment techniques such as precision-effect test and precision-effect estimate with standard errors (PET-PEESE) and selection models. These methods, implemented in JASP and R, allow researchers without programming experience to conduct state-of-the-art publication-bias-adjusted meta-analysis. In this tutorial, we demonstrate how to conduct a publication-bias-adjusted meta-analysis in JASP and R and interpret the results. First, we explain two frequentist bias-correction methods: PET-PEESE and selection models. Second, we introduce robust Bayesian meta-analysis, a Bayesian approach that simultaneously considers both PET-PEESE and selection models. We illustrate the methodology on an example data set, provide an instructional video ( https://bit.ly/pubbias ) and an R-markdown script ( https://osf.io/uhaew/ ), and discuss the interpretation of the results. Finally, we include concrete guidance on reporting the meta-analytic results in an academic article.
Publication Bias: A Detailed Analysis of Experiments Published in ESEM

Rolando P. Reyes,Óscar Dieste,Efraín R. Fonseca C.,Natalia Juristo

DOI: https://doi.org/10.48550/arXiv.2106.12533

2021-06-23

Software Engineering

Abstract:Background: Publication bias is the failure to publish the results of a study based on the direction or strength of the study findings. The existence of publication bias is firmly established in areas like medical research. Recent research suggests the existence of publication bias in Software Engineering. Aims: Finding out whether experiments published in the International Workshop on Empirical Software Engineering and Measurement (ESEM) are affected by publication bias. Method: We review experiments published in ESEM. We also survey with experimental researchers to triangulate our findings. Results: ESEM experiments do not define hypotheses and frequently perform multiple testing. One-tailed tests have a slightly higher rate of achieving statistically significant results. We could not find other practices associated with publication bias. Conclusions: Our results provide a more encouraging perspective of SE research than previous research: (1) ESEM publications do not seem to be strongly affected by biases and (2) we identify some practices that could be associated with p-hacking, but it is more likely that they are related to the conduction of exploratory research.
Sensitivity analysis for publication bias in meta‐analyses

Maya B. Mathur,Tyler J. VanderWeele

DOI: https://doi.org/10.1111/rssc.12440

2020-08-28

Abstract:We propose sensitivity analyses for publication bias in meta‐analyses. We consider a publication process such that 'statistically significant' results are more likely to be published than negative or "non‐significant" results by an unknown ratio, η. Our proposed methods also accommodate some plausible forms of selection based on a study's standard error. Using inverse probability weighting and robust estimation that accommodates non‐normal population effects, small meta‐analyses, and clustering, we develop sensitivity analyses that enable statements such as 'For publication bias to shift the observed point estimate to the null, "significant" results would need to be at least 30 fold more likely to be published than negative or "non‐significant" results'. Comparable statements can be made regarding shifting to a chosen non‐null value or shifting the confidence interval. To aid interpretation, we describe empirical benchmarks for plausible values of η across disciplines. We show that a worst‐case meta‐analytic point estimate for maximal publication bias under the selection model can be obtained simply by conducting a standard meta‐analysis of only the negative and 'non‐significant' studies; this method sometimes indicates that no amount of such publication bias could 'explain away' the results. We illustrate the proposed methods by using real meta‐analyses and provide an R package: PublicationBias.

statistics & probability
Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods

Evan C. Carter,Felix D. Schönbrodt,Will M. Gervais,Joseph Hilgard

DOI: https://doi.org/10.1177/2515245919847196

2019-06-01

Advances in Methods and Practices in Psychological Science

Abstract:Publication bias and questionable research practices in primary research can lead to badly overestimated effects in meta-analysis. Methodologists have proposed a variety of statistical approaches to correct for such overestimation. However, it is not clear which methods work best for data typically seen in psychology. Here, we present a comprehensive simulation study in which we examined how some of the most promising meta-analytic methods perform on data that might realistically be produced by research in psychology. We simulated several levels of questionable research practices, publication bias, and heterogeneity, and used study sample sizes empirically derived from the literature. Our results clearly indicated that no single meta-analytic method consistently outperformed all the others. Therefore, we recommend that meta-analysts in psychology focus on sensitivity analyses—that is, report on a variety of methods, consider the conditions under which these methods fail (as indicated by simulation studies such as ours), and then report how conclusions might change depending on which conditions are most plausible. Moreover, given the dependence of meta-analytic methods on untestable assumptions, we strongly recommend that researchers in psychology continue their efforts to improve the primary literature and conduct large-scale, preregistered replications. We provide detailed results and simulation code at https://osf.io/rf3ys and interactive figures at http://www.shinyapps.org/apps/metaExplorer/ .
Using selection models to assess sensitivity to publication bias: A tutorial and call for more routine use

Maximilian Maier,Tyler J. VanderWeele,Maya B. Mathur

DOI: https://doi.org/10.1002/cl2.1256

2022-07-20

Campbell Systematic Reviews

Abstract:In meta‐analyses, it is critical to assess the extent to which publication bias might have compromised the results. Classical methods based on the funnel plot, including Egger's test and Trim‐and‐Fill, have become the de facto default methods to do so, with a large majority of recent meta‐analyses in top medical journals (85%) assessing for publication bias exclusively using these methods. However, these classical funnel plot methods have important limitations when used as the sole means of assessing publication bias: they essentially assume that the publication process favors large point estimates for small studies and does not affect the largest studies, and they can perform poorly when effects are heterogeneous. In light of these limitations, we recommend that meta‐analyses routinely apply other publication bias methods in addition to or instead of classical funnel plot methods. To this end, we describe how to use and interpret selection models. These methods make the often more realistic assumption that publication bias favors "statistically significant" results, and the methods also directly accommodate effect heterogeneity. Selection models have been established for decades in the statistics literature and are supported by user‐friendly software, yet remain rarely reported in many disciplines. We use a previously published meta‐analysis to demonstrate that selection models can yield insights that extend beyond those provided by funnel plot methods, suggesting the importance of establishing more comprehensive reporting practices for publication bias assessment.
A modified regression method to test publication bias in meta-analyses with binary outcomes

Zhi-Chao Jin,Cheng Wu,Xiao-Hua Zhou,Jia He

DOI: https://doi.org/10.1186/1471-2288-14-132

2014-12-01

BMC Medical Research Methodology

Abstract:BackgroundThe tendency towards publication bias is greater for observational studies than for randomized clinical trials. Several statistical methods have been developed to test the publication bias. However, almost all existing methods exhibit rather low power or have inappropriate type I error rates.MethodsWe propose a modified regression method, which used a smoothed variance to estimate the precision of a study, to test for publication bias in meta-analyses of observational studies. A comprehensive simulation study is carried out, and a real-world example is considered.ResultsThe simulation results indicate that the performance of tests varies with the number of included studies, level of heterogeneity, event rates, and sample size ratio between two groups. Neither the existing tests nor the newly developed method is particularly powerful in all simulation scenarios. However, our proposed method has a more robust performance across different settings. In the presence of heterogeneity, the arcsine-Thompson test is a suitable alternative, and Peters’ test can be considered as a complementary method when mild or no heterogeneity is present.ConclusionsSeveral factors should be taken into consideration when employing asymmetry tests for publication bias. Based on our simulation results, we provide a concise table to show the appropriate use of regression methods to test for publication bias based on our simulation results.

health care sciences & services
Testing for Publication Bias in Diagnostic Meta-Analysis: A Simulation Study

Paul-Christian Bürkner,Philipp Doebler

DOI: https://doi.org/10.48550/arXiv.2211.12538

2022-11-22

Methodology

Abstract:The present study investigates the performance of several statistical tests to detect publication bias in diagnostic meta-analysis by means of simulation. While bivariate models should be used to pool data from primary studies in diagnostic meta-analysis, univariate measures of diagnostic accuracy are preferable for the purpose of detecting publication bias. In contrast to earlier research, which focused solely on the diagnostic odds ratio or its logarithm ($\ln\omega$), the tests are combined with four different univariate measures of diagnostic accuracy. For each combination of test and univariate measure, both type I error rate and statistical power are examined under diverse conditions. The results indicate that tests based on linear regression or rank correlation cannot be recommended in diagnostic meta-analysis, because type I error rates are either inflated or power is too low, irrespective of the applied univariate measure. In contrast, the combination of trim and fill and $\ln\omega$ has non-inflated or only slightly inflated type I error rates and medium to high power, even under extreme circumstances (at least when the number of studies per meta-analysis is large enough). Therefore, we recommend the application of trim and fill combined with $\ln\omega$ to detect funnel plot asymmetry in diagnostic meta-analysis. Please cite this paper as published in Statistics in Medicine (https://doi.org/10.1002/sim.6177).
A Generalized Publication Bias Model

Peter H. Schonemann,Jeffrey D. Scargle

DOI: https://doi.org/10.48550/arXiv.0808.1588

2008-08-11

Methodology

Abstract:Scargle (2000) has discussed Rosenthal and Rubin's (1978) "fail-safe number" (FSN) method for estimating the number of unpublished studies in meta-analysis. He concluded that this FSN cannot possibly be correct because a central assumption the authors used conflicts with the very definition of publication bias. While this point has been made by others before (Elsahoff, 1978; Darlington, 1980; Thomas, 1985, Iyengar & Greenhouse, 1988), Scargle showed, by way of a simple 2-parameter model, how far off Rosenthal & Rubin' s estimate can be in practice. However, his results relied on the assumption that the decision variable is normally distributed with zero mean. In this case the ratio of unpublished to published papers is large only in a tiny region of the parameter plane. Building on these results, we now show that (1) Replacing densities with probability masses greatly simplifies Scargle's derivations and permits an explicit statement of the relation between the probability alpha of Type I errors and the step-size beta; (2) This result does not require any distribution assumptions; (3) The distinction between 1-sided and 2-sided rejection regions becomes immaterial; (4) This distribution-free approach leads to an immediate generalization to partitions involving more than two intervals, and thus covers more general selection functions.
Publication Bias and Dissemination of Clinical Research

C. B Begg,J. A Berlin

DOI: https://doi.org/10.1093/jnci/81.2.107

1989-01-18

JNCI Journal of the National Cancer Institute

Abstract:Publication bias is a widely recognized phenomenon that occurs because of the influence of study results on the chances of publication. Usually, studies with positive results are more likely to be published than studies with negative results, which leads to a preponderance of false-positive results in the literature. Empiric studies have demonstrated that the induced bias is large and can have a serious impact on meta-analyses, in which data from several studies are aggregated, as well as on informal reviews. The problem is deeply embedded in current research practice, which encourages demonstration of statistical significance to "prove" theories, and one of its causes is the pressure to publish extensively that is an integral part of the competition for academic promotion. Serious efforts to reduce this problem will involve restructuring the process by which study results are disseminated, changing editorial policies, and altering the style and methods of statistical analysis.

oncology
Footprint of publication selection bias on meta‐analyses in medicine, environmental sciences, psychology, and economics

František Bartoš,Maximilian Maier,Eric‐Jan Wagenmakers,Franziska Nippold,Hristos Doucouliagos,John P. A. Ioannidis,Willem M. Otte,Martina Sladekova,Teshome K. Deresssa,Stephan B. Bruns,Daniele Fanelli,T. D. Stanley

DOI: https://doi.org/10.1002/jrsm.1703

2024-02-09

Research Synthesis Methods

Abstract:Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta‐analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta‐analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta‐analyses in environmental sciences and psychology, whereas meta‐analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine.

multidisciplinary sciences,mathematical & computational biology
Hybrid test for publication bias in meta-analysis

Lifeng Lin

DOI: https://doi.org/10.1177/0962280220910172

IF: 2.494

2020-04-15

Statistical Methods in Medical Research

Abstract:Publication bias frequently appears in meta-analyses when the included studies’ results (e.g., p-values) influence the studies’ publication processes. Some unfavorable studies may be suppressed from publication, so the meta-analytic results may be biased toward an artificially favorable direction. Many statistical tests have been proposed to detect publication bias in recent two decades. However, they often make dramatically different assumptions about the cause of publication bias; therefore, they are usually powerful only in certain cases that support their particular assumptions, while their powers may be fairly low in many other cases. Although several simulation studies have been carried out to compare different tests’ powers under various situations, it is typically infeasible to justify the exact mechanism of publication bias in a real-world meta-analysis and thus select the corresponding optimal publication bias test. We introduce a hybrid test for publication bias by synthesizing various tests and incorporating their benefits, so that it maintains relatively high powers across various mechanisms of publication bias. The superior performance of the proposed hybrid test is illustrated using simulation studies and three real-world meta-analyses with different effect sizes. It is compared with many existing methods, including the commonly used regression and rank tests, and the trim-and-fill method.

health care sciences & services,medical informatics,mathematical & computational biology,statistics & probability
Estimating the change in meta-analytic effect size estimates after the application of publication bias adjustment methods.

Martina Sladekova,Lois E. A. Webb,Andy P. Field

DOI: https://doi.org/10.1037/met0000470

IF: 10.929

2022-04-21

Psychological Methods

Abstract:Publication bias poses a challenge for accurately synthesizing research findings using meta-analysis. A number of statistical methods have been developed to combat this problem by adjusting the meta-analytic estimates. Previous studies tended to apply these methods without regard to optimal conditions for each method's performance. The present study sought to estimate the typical effect size attenuation of these methods when they are applied to real meta-analytic data sets that match the conditions under which each method is known to remain relatively unbiased (such as sample size, level of heterogeneity, population effect size, and the level of publication bias). Four-hundred and 33 data sets from 90 articles published in psychology journals were reanalyzed using a selection of publication bias adjustment methods. The downward adjustment found in our sample was minimal, with greatest identified attenuation of b = -.032, 95% highest posterior density interval (HPD) ranging from -.055 to -.009, for the precision effect test (PET). Some methods tended to adjust upward, and this was especially true for data sets with a sample size smaller than 10. We propose that researchers should seek to explore the full range of plausible estimates for the effects they are studying and note that these methods may not be able to combat bias in small samples (with less than 10 primary studies). We argue that although the effect size attenuation we found tended to be minimal, this should not be taken as an indication of low levels of publication bias in psychology. We discuss the findings with reference to new developments in Bayesian methods for publication bias adjustment, and the recent methodological reforms in psychology. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

psychology, multidisciplinary
Univariate Description and Bivariate Statistical Inference: the First Step Delving into Data

Zhongheng Zhang

DOI: https://doi.org/10.21037/atm.2016.02.11

IF: 3.616

2016-01-01

Annals of Translational Medicine

Abstract:In observational studies, the first step is usually to explore data distribution and the baseline differences between groups. Data description includes their central tendency (e.g., mean, median, and mode) and dispersion (e.g., standard deviation, range, interquartile range). There are varieties of bivariate statistical inference methods such as Student's t-test, Mann-Whitney U test and Chi-square test, for normal, skews and categorical data, respectively. The article shows how to perform these analyses with R codes. Furthermore, I believe that the automation of the whole workflow is of paramount importance in that (I) it allows for others to repeat your results; (II) you can easily find out how you performed analysis during revision; (III) it spares data input by hand and is less error-prone; and (IV) when you correct your original dataset, the final result can be automatically corrected by executing the codes. Therefore, the process of making a publication quality table incorporating all abovementioned statistics and P values is provided, allowing readers to customize these codes to their own needs.
A regression-based method for detecting publication bias in multivariate meta-analysis

Chuan Hong,Jing Zhang,Yang Li,Elena Elia,Richard Riley,Yong Chen

DOI: https://doi.org/10.48550/arXiv.2002.04775

2020-02-12

Methodology

Abstract:Publication bias occurs when the publication of research results depends not only on the quality of the research but also on its nature and direction. The consequence is that published studies may not be truly representative of all valid studies undertaken, and this bias may threaten the validity of systematic reviews and meta-analyses - on which evidence-based medicine increasingly relies. Multivariate meta-analysis has recently received increasing attention for its ability reducing potential bias and improving statistical efficiency by borrowing information across outcomes. However, detecting and accounting for publication bias are more challenging in multivariate meta-analysis setting because some studies may be completely unpublished whereas some studies may selectively report part of multiple outcomes. In this paper, we propose a score test for jointly testing publication bias for multiple outcomes, which is novel to the multivariate setting. The proposed test is a natural multivariate extension of the univariate Egger's test, and can handle the above mentioned scenarios simultaneously, It accounts for correlations among multivariate outcomes, while allowing different types of outcomes, and can borrow information across outcomes. The proposed test is shown to be more powerful than the Egger's test, Begg's test and Trim and Fill method through simulation studies. Two data analyses are given to illustrate the performance of the proposed test in practice.
Measuring, visualizing, and diagnosing reference bias with biastools

Mao-Jan Lin,Sheila Iyer,Nae-Chyun Chen,Ben Langmead

DOI: https://doi.org/10.1186/s13059-024-03240-8

IF: 17.906

2024-04-21

Genome Biology

Abstract:Many bioinformatics methods seek to reduce reference bias, but no methods exist to comprehensively measure it. Biastools analyzes and categorizes instances of reference bias. It works in various scenarios: when the donor's variants are known and reads are simulated; when donor variants are known and reads are real; and when variants are unknown and reads are real. Using biastools , we observe that more inclusive graph genomes result in fewer biased sites. We find that end-to-end alignment reduces bias at indels relative to local aligners. Finally, we use biastools to characterize how T2T references improve large-scale bias.

genetics & heredity,biotechnology & applied microbiology

Software for Publication Bias

Examining publication bias—a simulation-based evaluation of statistical tests on publication bias

Publication Bias in the Organizational Sciences

The power functions of Begg's and Egger's tests for publication bias

Systematic Evaluation and Comparison of Statistical Tests for Publication Bias

Empirical Comparison of Publication Bias Tests in Meta-Analysis

Adjusting for Publication Bias in JASP and R: Selection Models, PET-PEESE, and Robust Bayesian Meta-Analysis

Publication Bias: A Detailed Analysis of Experiments Published in ESEM

Sensitivity analysis for publication bias in meta‐analyses

Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods

Using selection models to assess sensitivity to publication bias: A tutorial and call for more routine use

A modified regression method to test publication bias in meta-analyses with binary outcomes

Testing for Publication Bias in Diagnostic Meta-Analysis: A Simulation Study

A Generalized Publication Bias Model

Publication Bias and Dissemination of Clinical Research

Footprint of publication selection bias on meta‐analyses in medicine, environmental sciences, psychology, and economics

Hybrid test for publication bias in meta-analysis

Estimating the change in meta-analytic effect size estimates after the application of publication bias adjustment methods.

Univariate Description and Bivariate Statistical Inference: the First Step Delving into Data

A regression-based method for detecting publication bias in multivariate meta-analysis

Measuring, visualizing, and diagnosing reference bias with biastools