Abstract:Abstract Background The complexity of public health interventions create challenges in evaluating their effectiveness. There have been huge advancements in quantitative evidence synthesis methods development (including meta-analysis) for dealing with heterogeneity of intervention effects, inappropriate ‘lumping’ of interventions, adjusting for different populations and outcomes and the inclusion of various study types. Growing awareness of the importance of using all available evidence has led to the publication of guidance documents for implementing methods to improve decision making by answering policy relevant questions. Methods The first part of this paper reviews the methods used to synthesise quantitative effectiveness evidence in public health guidelines by the National Institute for Health and Care Excellence (NICE) that had been published or updated since the previous review in 2012 until the 19th August 2019.The second part of this paper provides an update of the statistical methods and explains how they address issues related to evaluating effectiveness evidence of public health interventions. Results The proportion of NICE public health guidelines that used a meta-analysis as part of the synthesis of effectiveness evidence has increased since the previous review in 2012 from 23% (9 out of 39) to 31% (14 out of 45). The proportion of NICE guidelines that synthesised the evidence using only a narrative review decreased from 74% (29 out of 39) to 60% (27 out of 45).An application in the prevention of accidents in children at home illustrated how the choice of synthesis methods can enable more informed decision making by defining and estimating the effectiveness of more distinct interventions, including combinations of intervention components, and identifying subgroups in which interventions are most effective. Conclusions Despite methodology development and the publication of guidance documents to address issues in public health intervention evaluation since the original review, NICE public health guidelines are not making full use of meta-analysis and other tools that would provide decision makers with fuller information with which to develop policy. There is an evident need to facilitate the translation of the synthesis methods into a public health context and encourage the use of methods to improve decision making.

Context and Implications Document for: Ongoing developments in meta-analytic and quantitative synthesis methods: Broadening the types of research questions that can be addressed

Synthesizing and presenting findings using other methods

Innovations in framework synthesis as a systematic review method

What do meta-analysts need in primary studies? Guidelines and the SEMI checklist for facilitating cumulative knowledge

Second-order synthesis of meta-analytic studies in applied linguistics (1998–2021)

What meta‐research has taught us about research and changes to research practices

Scoping reviews: reinforcing and advancing the methodology and application

Contexts of diffusion: Adoption of research synthesis in Social Work and Women's Studies

The use of ‘PICO for synthesis’ and methods for synthesis without meta-analysis: protocol for a survey of current practice in systematic reviews of health interventions

Quantitative evidence synthesis: a practical guide on meta-analysis, meta-regression, and publication bias tests for environmental sciences

Methodological Issues in Combining Diverse Study Types in Systematic Reviews

Methodological Guidance Paper: High-Quality Meta-Analysis in a Systematic Review

A systematic review of meta-analyses in second language research: current practices, issues, and recommendations

Enriching meta-analyses through scoping review, bibliometrics, and alternative impact metrics: Visualizing study characteristics, hidden risk of bias, societal influence, and research translation

Systematically Searching Empirical Literature in the Social Sciences: Results from Two Meta-Analyses Within the Domain of Education

A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines

Towards more scientific meta-analyses

Methods to Address Confounding and Other Biases in Meta-Analyses: Review and Recommendations

Fragile Findings, Complex Conclusions, and Meta-Analysis in Special Education

Developing, Testing, or Extending Theory with Meta-analyses

A Call For Cautious Interpretation Of Meta-Analytic Reviews