Abstract:Abstract Background The complexity of public health interventions create challenges in evaluating their effectiveness. There have been huge advancements in quantitative evidence synthesis methods development (including meta-analysis) for dealing with heterogeneity of intervention effects, inappropriate ‘lumping’ of interventions, adjusting for different populations and outcomes and the inclusion of various study types. Growing awareness of the importance of using all available evidence has led to the publication of guidance documents for implementing methods to improve decision making by answering policy relevant questions. Methods The first part of this paper reviews the methods used to synthesise quantitative effectiveness evidence in public health guidelines by the National Institute for Health and Care Excellence (NICE) that had been published or updated since the previous review in 2012 until the 19th August 2019.The second part of this paper provides an update of the statistical methods and explains how they address issues related to evaluating effectiveness evidence of public health interventions. Results The proportion of NICE public health guidelines that used a meta-analysis as part of the synthesis of effectiveness evidence has increased since the previous review in 2012 from 23% (9 out of 39) to 31% (14 out of 45). The proportion of NICE guidelines that synthesised the evidence using only a narrative review decreased from 74% (29 out of 39) to 60% (27 out of 45).An application in the prevention of accidents in children at home illustrated how the choice of synthesis methods can enable more informed decision making by defining and estimating the effectiveness of more distinct interventions, including combinations of intervention components, and identifying subgroups in which interventions are most effective. Conclusions Despite methodology development and the publication of guidance documents to address issues in public health intervention evaluation since the original review, NICE public health guidelines are not making full use of meta-analysis and other tools that would provide decision makers with fuller information with which to develop policy. There is an evident need to facilitate the translation of the synthesis methods into a public health context and encourage the use of methods to improve decision making.

Synthesizing and presenting findings using other methods

Synthesising results of meta-analyses to inform policy: a comparison of fast-track methods

A further use for the Harvest plot: a novel method for the presentation of data synthesis

The use of ‘PICO for synthesis’ and methods for synthesis without meta-analysis: protocol for a survey of current practice in systematic reviews of health interventions

Towards more scientific meta-analyses

A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines

Metaanalysis - Literature Synthesis or Effect-Size Surface Estimation

Empirical Comparisons of 12 Meta-analysis Methods for Synthesizing Proportions of Binary Outcomes

Sequential Bayesian Data Synthesis for Mediation and Regression Analysis

Synthesis methods used to combine observational studies and randomised trials in published meta-analyses

Second-order synthesis of meta-analytic studies in applied linguistics (1998–2021)

Quantitative evidence synthesis: a practical guide on meta-analysis, meta-regression, and publication bias tests for environmental sciences

Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline

Enriching meta-analyses through scoping review, bibliometrics, and alternative impact metrics: Visualizing study characteristics, hidden risk of bias, societal influence, and research translation

The effect direction plot revisited: Application of the 2019 Cochrane Handbook guidance on alternative synthesis methods

What do meta-analysts need in primary studies? Guidelines and the SEMI checklist for facilitating cumulative knowledge

Fragile Findings, Complex Conclusions, and Meta-Analysis in Special Education

Robust Point and Variance Estimation for Meta‐analyses with Selective Reporting and Dependent Effect Sizes

Evidence synthesis summary formats for clinical guideline development group members: a mixed-methods systematic review protocol

Context and Implications Document for: Ongoing developments in meta-analytic and quantitative synthesis methods: Broadening the types of research questions that can be addressed

Innovations in framework synthesis as a systematic review method