Decision makers need constantly updated evidence synthesis

Julian Elliott,Rebecca Lawrence,Jan C. Minx,Olufemi T. Oladapo,Philippe Ravaud,Britta Tendal Jeppesen,James Thomas,Tari Turner,Per Olav Vandvik,Jeremy M. Grimshaw

DOI: https://doi.org/10.1038/d41586-021-03690-1

IF: 64.8

2021-12-15

Nature

Abstract:Fund and use ‘living’ reviews of the latest data to steer research, practice and policy.

multidisciplinary sciences

What problem does this paper attempt to address?

Need for evidence synthesis for quality control of healthcare decision-making

Brijesh Sathian,Edwin van Teijlingen,Israel Junior Borges do Nascimento,Mahalaqua Nazli Khatib,Indrajit Banerjee,Padam Simkhada,Russell Kabir,Hanadi Al Hamad

DOI: https://doi.org/10.3126/nje.v13i3.61004

2023-12-23

Abstract:Systematic reviews that are out-of-date delay policymaking, create controversy, and can erode trust in research. To avoid this issue, it is preferable to keep summaries of the study evidence. Living evidence is a synthesis approach that provides up-to-date rigorous research evidence summaries to decision-makers. This strategy is particularly useful in rapidly expanding research domains, uncertain existing evidence, and new research that may impact policy or practice, ensuring that physicians have access to the most recent evidence. Addressing global challenges - ranging from public health crises to climate change or political instability - requires evidence-based judgements. An obsolete, biased, or selective information poses risks of poor decisions and resource misallocation. The relatively nascent practice of living evidence proves invaluable in maintaining continuous interest and team engagement. The concept of living evidence has been particularly relevant during the COVID-19 pandemic due to the rapidly evolving nature of the virus, the urgent need for timely information, and the continuous emergence of new research findings. Although the COVID-19 pandemic accelerated the adoption of evidence systems, researchers and funders of research should rigorously test the living-evidence model across diverse domains to further advance and optimize its methodology.
Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap

Julian H Elliott, Tari Turner, Ornella Clavisi, James Thomas, Julian PT Higgins, Chris Mavergames, Russell L Gruen

2014-02-18

Abstract:Julian Elliott and colleagues discuss how the current inability to keep systematic reviews up-to-date hampers the translation of knowledge into action. They propose living systematic reviews as a contribution to evidence synthesis to enhance the accuracy and utility of health evidence.
Living systematic review: 1. Introduction—the why, what, when, and how

Julian H Elliott, Anneliese Synnot, Tari Turner, Mark Simmonds, Elie A Akl, Steve McDonald, Georgia Salanti, Joerg Meerpohl, Harriet MacLehose, John Hilton, David Tovey, Ian Shemilt, James Thomas, Thomas Agoritsas, Caroline Perron, Elie Akl, Rebecca Hodder, Charlotte Pestridge, Lauren Albrecht, Tanya Horsley, Joanne Platt, Rebecca Armstrong, Phi Hung Nguyen, Robert Plovnick, Anneliese Arno, Noah Ivers, Gail Quinn, Agnes Au, Renea Johnston, Gabriel Rada, Matthew Bagg, Arwel Jones, Philippe Ravaud, Catherine Boden, Lara Kahale, Bernt Richter, Isabelle Boisvert, Homa Keshavarz, Rebecca Ryan, Linn Brandt, Stephanie A Kolakowsky-Hayner, Dina Salama, Alexandra Brazinova, Sumanth Kumbargere Nagraj, Rachelle Buchbinder, Toby Lasserson, Lina Santaguida, Chris Champion, Rebecca Lawrence, Nancy Santesso, Jackie Chandler, Zbigniew Les, Holger J Schünemann, Andreas Charidimou, Stefan Leucht, Roger Chou, Nicola Low, Diana Sherifali, Rachel Churchill, Andrew Maas, Reed Siemieniuk, Maryse C Cnossen, Marie-Joelle Cossi, Malcolm Macleod, Nicole Skoetz, Michel Counotte, Iain Marshall, Karla Soares-Weiser, Samantha Craigie, Rachel Marshall, Velandai Srikanth, Philipp Dahm, Nicole Martin, Katrina Sullivan, Alanna Danilkewich, Laura Martínez García, Kristen Danko, Chris Mavergames, Mark Taylor, Emma Donoghue, Lara J Maxwell, Kris Thayer, Corinna Dressler, James McAuley, Cathy Egan, Roger Tritton, Julian Elliott, Joanne McKenzie, Guy Tsafnat, Sarah A Elliott, Peter Tugwell, Itziar Etxeandia, Bronwen Merner, Alexis Turgeon, Robin Featherstone, Stefania Mondello, Ruth Foxlee, Richard Morley, Gert van Valkenhoef, Paul Garner, Marcus Munafo, Per Vandvik, Martha Gerrity, Zachary Munn, Byron Wallace, Paul Glasziou, Melissa Murano, Sheila A Wallace, Sally Green, Kristine Newman, Chris Watts, Jeremy Grimshaw, Robby Nieuwlaat, Laura Weeks, Kurinchi Gurusamy, Adriani Nikolakopoulou, Aaron Weigl, Neal Haddaway, Anna Noel-Storr, George Wells, Lisa Hartling, Annette O'Connor, Wojtek Wiercioch, Jill Hayden, Matthew Page, Luke Wolfenden, Mark Helfand, Manisha Pahwa, Juan José Yepes Nuñez, Julian Higgins, Jordi Pardo Pardo, Jennifer Yost, Sophie Hill, Leslea Pearson

2017-11-01

Abstract:Systematic reviews are difficult to keep up to date, but failure to do so leads to a decay in review currency, accuracy, and utility. We are developing a novel approach to systematic review updating termed “Living systematic review” (LSR): systematic reviews that are continually updated, incorporating relevant new evidence as it becomes available. LSRs may be particularly important in fields where research evidence is emerging rapidly, current evidence is uncertain, and new research may change policy or practice decisions. We hypothesize that a continual approach to updating will achieve greater currency and validity, and increase the benefits to end users, with feasible resource requirements over time.
Flexible synthesis to deliver relevant evidence for decision-making

Philip Anthony Martin,Alec Philip Christie,Gorm Shackelford,Amelia Hood,Shengyu Wang,Bo Li,William Morgan,Madeleine Lee,David C. Aldridge,William J Sutherland

DOI: https://doi.org/10.31219/osf.io/3t9jy

2022-01-01

Abstract:The use of systematic reviews to collate and summarise evidence is considered one of the great intellectual achievements of recent times. Evidence synthesis has had far-reaching impacts and has helped to inform decision-making in multiple fields. However, it also faces three problems: (i) reviews cannot be relevant to everyone, hampering decision-making, (ii) multiple conflicting review are often produced, and (iii) reviews can become out-of-date rapidly. Here we present a solution to these problems that we term 'dynamic meta-analysis.' With a tool we have developed, users can filter and weight evidence depending on their interests to produce analyses that are highly relevant to their needs.
Future of evidence ecosystem series: 2. current opportunities and need for better tools and methods

Perrine Créquit,Isabelle Boutron,Joerg Meerpohl,Hywel C Williams,Jonathan Craig,Philippe Ravaud

DOI: https://doi.org/10.1016/j.jclinepi.2020.01.023

Abstract:To become user driven and more useful for decision-making, the current evidence synthesis ecosystem requires significant changes (Paper 1. Future of evidence ecosystem series). Reviewers have access to new sources of data (clinical trial registries, protocols, and clinical study reports from regulatory agencies or pharmaceutical companies) for more information on randomized control trials. With all these newly available data, the management of multiple and scattered trial reports is even more challenging. New types of data are also becoming available: individual patient data and routinely collected data. With the increasing number of diverse sources to be searched and the amount of data to be extracted, the process needs to be rethought. New approaches and tools, such as automation technologies and crowdsourcing, should help accelerate the process. The implementation of these new approaches and methods requires a substantial rethinking and redesign of the current evidence synthesis ecosystem. The concept of a "living" evidence synthesis enterprise, with living systematic review and living network meta-analysis, has recently emerged. Such an evidence synthesis ecosystem implies conceptualizing evidence synthesis as a continuous process built around a clinical question of interest and no longer as a small team independently answering a specific clinical question at a single point in time.
A practical guide to living evidence: Reducing the knowledge-to-practice gap

Sabine M Allida,Maree L Hackett,Richard Lindley,Kelvin Hill,Caleb Ferguson

DOI: https://doi.org/10.1093/eurjcn/zvae048

2024-04-08

European Journal of Cardiovascular Nursing

Abstract:Abstract Living evidence involves continuous evidence surveillance to incorporate new relevant evidence into systematic reviews and clinical practice guideline recommendations as soon as it becomes available. Thus, living evidence may improve the timeliness of recommendation updates and reduce the knowledge-to-practice gap. When considering a living evidence model, several processes and practical aspects need to be explored. Some of these include identifying the need for a living evidence model, funding, governance structure, time, team skills and capabilities, frequency of updates, approval and endorsement and publication and dissemination.

nursing,cardiac & cardiovascular systems
A fresh approach to evidence synthesis

William J Sutherland, Claire FR Wordley

2018-06-21

Abstract:Systematic reviews have transformed medicine, but a more cost-effective means of appraisal is needed for fields in which data are sparse and patchy, argue William J. Sutherland and Claire F. R. Wordley.
A narrative review of recent tools and innovations toward automating living systematic reviews and evidence syntheses

Lena Schmidt,Mark Sinyor,Roger T Webb,Christopher Marshall,Duleeka Knipe,Emily C Eyles,Ann John,David Gunnell,Julian P T Higgins

DOI: https://doi.org/10.1016/j.zefq.2023.06.007

Abstract:Living reviews are an increasingly popular research paradigm. The purpose of a 'living' approach is to allow rapid collation, appraisal and synthesis of evolving evidence on an important research topic, enabling timely influence on patient care and public health policy. However, living reviews are time- and resource-intensive. The accumulation of new evidence and the possibility of developments within the review's research topic can introduce unique challenges into the living review workflow. To investigate the potential of software tools to support living systematic or rapid reviews, we present a narrative review informed by an examination of tools contained on the Systematic Review Toolbox website. We identified 11 tools with relevant functionalities and discuss the important features of these tools with respect to different steps of the living review workflow. Four tools (NestedKnowledge, SWIFT-ActiveScreener, DistillerSR, EPPI-Reviewer) covered multiple, successive steps of the review process, and the remaining tools addressed specific components of the workflow, including scoping and protocol formulation, reference retrieval, automated data extraction, write-up and dissemination of data. We identify several ways in which living reviews can be made more efficient and practical. Most of these focus on general workflow management, or automation through artificial intelligence and machine-learning, in the screening process. More sophisticated uses of automation mostly target living rapid reviews to increase the speed of production or evidence maps to broaden the scope of the map. We use a case study to highlight some of the barriers and challenges to incorporating tools into the living review workflow and processes. These include increased workload, the need for organisation, ensuring timely dissemination and challenges related to the development of bespoke automation tools to facilitate the review process. We describe how current end-user tools address these challenges, and which knowledge gaps remain that could be addressed by future tool development. Dedicated web presences for automatic dissemination of in-progress evidence updates, rather than solely relying on peer-reviewed journal publications, help to make the effort of a living evidence synthesis worthwhile. Despite offering basic living review functionalities, existing end-user tools could be further developed to be interoperable with other tools to support multiple workflow steps seamlessly, to address broader automatic evidence retrieval from a larger variety of sources, and to improve dissemination of evidence between review updates.
A living critical interpretive synthesis to yield a framework on the production and dissemination of living evidence syntheses for decision-making

Cristián Mansilla,Qi Wang,Thomas Piggott,Peter Bragge,Kerry Waddell,Gordon Guyatt,Arthur Sweetman,John N. Lavis

DOI: https://doi.org/10.1186/s13012-024-01396-2

2024-09-30

Implementation Science

Abstract:The COVID-19 pandemic has had an unprecedented impact in the global research production and has also increased research waste. Living evidence syntheses (LESs) seek to regularly update a body of evidence addressing a specific question. During the COVID-19 pandemic, the production and dissemination of LESs emerged as a cornerstone of the evidence infrastructure. This critical interpretive synthesis answers the questions: What constitutes an LES to support decision-making?; when should one be produced, updated, and discontinued?; and how should one be disseminated?

health care sciences & services,health policy & services
SHOW ME the evidence: Features of an approach to reliably deliver research evidence to those who need it

John N Lavis,Jeremy M Grimshaw,Ruth Stewart,Julian Elliott,Will Moy,Joerg J Meerpohl

DOI: https://doi.org/10.1002/14651858.ed000170

IF: 8.4

2024-11-22

Cochrane Database of Systematic Reviews

Abstract:The world is poised for a step‐change improvement in how we use evidence to address societal challenges. Given the speed at which plans are being made to support this once‐in‐a‐generation transformation, the Implementation Council of the Global Commission on Evidence to Address Societal Challenges developed a working version of the features of an approach to reliably getting research evidence to those who need it and achieved consensus among leaders from the Implementation Council, as well as the Alliance for Living Evidence (Alive) Council and Evidence Synthesis International (ESI). Drawing an acronym from the first letter of each of the six features, the 'SHOW ME the evidence' features are as follows. The 100+ contributing authors from across the 'evidence synthesis and support' world want to ensure that our future plans are firmly rooted in an agreed‐upon summary of all we have learned together over these past four or so years, and to signal a mutual accountability among many of the key players involved in providing evidence support that we will each do our part in delivering on the promise that motivates these plans. Given that much of the momentum for transformation is currently focused on living evidence syntheses and the infrastructure needed to support them, we give this form of evidence disproportionate focus here. An even more diverse set of partners should be engaged in designing and executing an inclusive process for the refinement or even reshaping of these features over time, as well as their ongoing operationalization. This includes more types of decision makers, those working with more forms of evidence, and funders, as well as even more contributors from across the Global South. Support systems locally that use many forms of research evidence to help address local priorities Every jurisdiction needs a reliable evidence‐support system to get whatever forms of evidence are needed to address a local priority into the hands of those who need it, when they need it, in whatever form they need it, and with any required caveats about its currency, quality, and local applicability [1]. Locally can mean nations as well as subnational jurisdictions like provinces and cities. It can mean formal regional groupings of countries like the European Union and informal regional groupings of small countries with shared challenges. It can also mean systems, like the health or social‐care system. The forms of evidence can include research evidence from the 'local' context (e.g. data analytics, evaluation, and behavioural or implementation research), research evidence from around the world (i.e. evidence synthesis), and other types of information (e.g. horizon scanning and people's lived experiences) and ways of knowing (e.g. Indigenous knowledge). Addressing a local priority is ideally informed by an understanding of a problem (and its causes and alternative ways of framing it), options to address the problem (including those already in use at a small scale), implementation considerations, and how to monitor implementation and evaluate impact. Research evidence can inform such understandings alongside political and social insights. Those who need research evidence can include government policy makers (from central agencies like Treasury, line departments like Education, and legislatures), organizational leaders (from both nongovernmental organizations and private companies), professionals (like nurses, teachers and veterinarians), and citizens (in the broadest sense of that term, and inclusive of undocumented individuals, as described in section 3.6 of the Global Evidence Commission report 2022). They also need enablers, culture and capacity for evidence use. Many decision makers need actionable insights from research evidence quickly when a 'window of opportunity' opens. Sometimes these windows are open for days, other times weeks, and rarely for longer. Evidence support can now work at the same speed as decision‐making processes. Some decision makers may want the evidence presented to them as 'best buys' (e.g. Global Education Evidence Advisory Panel), others by broad approach (e.g. Education Endowment Foundation), and still others by branded programme (e.g. IES What Works Clearinghouse). Applicability can mean both for local contexts and for groups in a range of contexts, including groups most affected by historical and acute inequities. Harmonized efforts globally that make it easier to learn from others around the world One aspect of evidence support that can now best be undertaken through harmonized efforts globally is to provide regularly updated summaries of what we have learned from around the world and how these findings vary by groups and contexts. 'Living evidence synthesis' is a relatively new approach to producing and maintaining these summaries [2]. The take‐up of this approach accelerated during the COVID‐19 pandemic -Abstract Truncated-

medicine, general & internal
ALL-IN meta-analysis: breathing life into living systematic reviews

Judith ter Schure,Peter Grünwald

DOI: https://doi.org/10.12688/f1000research.74223.1

2022-10-22

F1000Research

Abstract:Science is justly admired as a cumulative process ("standing on the shoulders of giants"), yet scientific knowledge is typically built on a patchwork of research contributions without much coordination. This lack of efficiency has specifically been addressed in clinical research by recommendations for living systematic reviews and against research waste. We propose to further those recommendations with ALL-IN meta-analysis: Anytime Live and Leading INterim meta-analysis. ALL-IN provides statistical methodology for a meta-analysis that can be updated at any time—reanalyzing after each new observation while retaining type-I error guarantees, live—no need to prespecify the looks, and leading—in the decisions on whether individual studies should be initiated, stopped or expanded, the meta-analysis can be the leading source of information. We illustrate the method for time-to-event data, showing how synthesizing data at interim stages of studies can increase efficiency when studies are slow in themselves to provide the necessary number of events for completion. The meta-analysis can be performed on interim data, but does not have to. The analysis design requires no information about the number of patients in trials or the number of trials eventually included. So it can breathe life into living systematic reviews, through better and simpler statistics, efficiency, collaboration and communication
Appraised evidence and decision making in clinical practice.

Chrysi Leliopoulou,Theodora Stroumpouki,Emmanouil Stafylarakis,Linda Collins

DOI: https://doi.org/10.1101/2024.11.15.24317373

2024-11-15

Abstract:Abstract Background: The use of evidence in decision making in clinical practice may vary significantly from practice to practice. Research suggests that clinicians may ignore research-based evidence in their practice or use research evidence indirectly. The aim of this systematic review is to explore using an online independent learning approach for clinicians making evidence-based decisions using research and evidence. Methodology: This systematic literature review was conducted across MEDLINE, PUBMED, SCIENCE Direct, APA PSYCINFO and CINHAL databases. The use of a search strategy employed to conduct searches on the individual database. Using the MMAT criteria by Hong et al., 2018) studies were screened and reviewed. Results: The findings from each study were thematically analysed and organised in the following thematic areas: (1) Searching for an answer to the clinical question or problem; (2) Expanding on the evidence to address the clinical question or problem; (3) Recording the available evidence to the clinical problem; (4) Validating the evidence, whether research-based or tacit knowledge; (5) Inspecting the evidence for its potential to improve current practice; (6) Cascading the evidence appropriately and possibly adopting the evidence in the decision; and (7) Evaluating the outcome of the clinical decision. Conclusions: This systematic literature review found there is a lack of an online independent learning approach for clinicians making evidence-based decision using research and evidence which highlights a gap in the literature. Keywords: screening evidence, decision-making, clinical practice inquiry.

Health Systems and Quality Improvement
Living systematic reviews: 2. Combining human and machine effort

James Thomas, Anna Noel-Storr, Iain Marshall, Byron Wallace, Steven McDonald, Chris Mavergames, Paul Glasziou, Ian Shemilt, Anneliese Synnot, Tari Turner, Julian Elliott, Thomas Agoritsas, John Hilton, Caroline Perron, Elie Akl, Rebecca Hodder, Charlotte Pestridge, Lauren Albrecht, Tanya Horsley, Joanne Platt, Rebecca Armstrong, Phi Hung Nguyen, Robert Plovnick, Anneliese Arno, Noah Ivers, Gail Quinn, Agnes Au, Renea Johnston, Gabriel Rada, Matthew Bagg, Arwel Jones, Philippe Ravaud, Catherine Boden, Lara Kahale, Bernt Richter, Isabelle Boisvert, Homa Keshavarz, Rebecca Ryan, Linn Brandt, Stephanie A Kolakowsky-Hayner, Dina Salama, Alexandra Brazinova, Sumanth Kumbargere Nagraj, Georgia Salanti, Rachelle Buchbinder, Toby Lasserson, Lina Santaguida, Chris Champion, Rebecca Lawrence, Nancy Santesso, Jackie Chandler, Zbigniew Les, Holger J Schünemann, Andreas Charidimou, Stefan Leucht, Roger Chou, Nicola Low, Diana Sherifali, Rachel Churchill, Andrew Maas, Reed Siemieniuk, Maryse C Cnossen, Harriet MacLehose, Mark Simmonds, Marie-Joelle Cossi, Malcolm Macleod, Nicole Skoetz, Michel Counotte, Karla Soares-Weiser, Samantha Craigie, Rachel Marshall, Velandai Srikanth, Philipp Dahm, Nicole Martin, Katrina Sullivan, Alanna Danilkewich, Laura Martínez García, Kristen Danko, Mark Taylor, Emma Donoghue, Lara J Maxwell, Kris Thayer, Corinna Dressler, James McAuley, Cathy Egan, Steve McDonald, Roger Tritton, Joanne McKenzie, Guy Tsafnat, Sarah A Elliott, Joerg Meerpohl, Peter Tugwell, Itziar Etxeandia, Bronwen Merner, Alexis Turgeon, Robin Featherstone, Stefania Mondello, Ruth Foxlee, Richard Morley, Gert van Valkenhoef, Paul Garner, Marcus Munafo, Per Vandvik, Martha Gerrity, Zachary Munn, Melissa Murano, Sheila A Wallace, Sally Green, Kristine Newman, Chris Watts, Jeremy Grimshaw, Robby Nieuwlaat, Laura Weeks, Kurinchi Gurusamy, Adriani Nikolakopoulou, Aaron Weigl, Neal Haddaway, George Wells, Lisa Hartling, Annette O'Connor, Wojtek Wiercioch, Jill Hayden, Matthew Page, Luke Wolfenden, Mark Helfand, Manisha Pahwa, Juan José Yepes Nuñez, Julian Higgins, Jordi Pardo Pardo, Jennifer Yost, Sophie Hill, Leslea Pearson

2017-11-01

Abstract:New approaches to evidence synthesis, which use human effort and machine automation in mutually reinforcing ways, can enhance the feasibility and sustainability of living systematic reviews. Human effort is a scarce and valuable resource, required when automation is impossible or undesirable, and includes contributions from online communities (“crowds”) as well as more conventional contributions from review authors and information specialists. Automation can assist with some systematic review tasks, including searching, eligibility assessment, identification and retrieval of full-text reports, extraction of data, and risk of bias assessment. Workflows can be developed in which human effort and machine automation can each enable the other to operate in more effective and efficient ways, offering substantial enhancement to the productivity of systematic reviews. This paper describes and discusses the potential …
Methods proposed for monitoring the implementation of evidence-based research: a cross-sectional study

Livia Puljak,Małgorzata M Bala,Joanna Zając,Tomislav Meštrović,Sandra Buttigieg,Mary Yanakoulia,Matthias Briel,Carole Lunny,Wiktoria Lesniak,Tina Poklepović Peričić,Pablo Alonso-Coello,Mike Clarke,Benjamin Djulbegovic,Gerald Gartlehner,Konstantinos Giannakou,Anne-Marie Glenny,Claire Glenton,Gordon Guyatt,Lars G Hemkens,John P A Ioannidis,Roman Jaeschke,Karsten Juhl Jørgensen,Carolina Castro Martins-Pfeifer,Ana Marušić,Lawrence Mbuagbaw,Jose Francisco Meneses Echavez,David Moher,Barbara Nussbaumer-Streit,Matthew J Page,Giordano Pérez-Gaxiola,Karen A Robinson,Georgia Salanti,Ian J Saldanha,Jelena Savović,James Thomas,Andrea C Tricco,Peter Tugwell,Joost van Hoof,Dawid Pieper,Małgorzata M. Bala,Lars G. Hemkens,John P.A. Ioannidis,Matthew J. Page,Karen A. Robinson,Ian J. Saldanha,Andrea C. Tricco

DOI: https://doi.org/10.1016/j.jclinepi.2024.111247

IF: 7.407

2024-04-01

Journal of Clinical Epidemiology

Abstract:OBJECTIVES: Evidence-based research (EBR) is the systematic and transparent use of prior research to inform a new study so that it answers questions that matter in a valid, efficient, and accessible manner. This study surveyed experts about existing (e.g., citation analysis) and new methods for monitoring EBR and collected ideas about implementing these methods.STUDY DESIGN AND SETTING: We conducted a cross-sectional study via an online survey between November 2022 and March 2023. Participants were experts from the fields of evidence synthesis and research methodology in health research. Open-ended questions were coded by recurring themes; descriptive statistics were used for quantitative questions.RESULTS: Twenty-eight expert participants suggested that citation analysis should be supplemented with content evaluation (not just what is cited but also in which context), content expert involvement, and assessment of the quality of cited systematic reviews. They also suggested that citation analysis could be facilitated with automation tools. They emphasized that EBR monitoring should be conducted by ethics committees and funding bodies before the research starts. Challenges identified for EBR implementation monitoring were resource constraints and clarity on responsibility for EBR monitoring.CONCLUSION: Ideas proposed in this study for monitoring the implementation of EBR can be used to refine methods and define responsibility but should be further explored in terms of feasibility and acceptability. Different methods may be needed to determine if the use of EBR is improving over time.

public, environmental & occupational health,health care sciences & services
Advancing the field of health systems research synthesis

Etienne V Langlois,Michael K Ranson,Till Bärnighausen,Xavier Bosch-Capblanch,Karen Daniels,Fadi El-Jardali,Abdul Ghaffar,Jeremy Grimshaw,Andy Haines,John N Lavis,Simon Lewin,Qingyue Meng,Sandy Oliver,Tomás Pantoja,Sharon Straus,Ian Shemilt,David Tovey,Peter Tugwell,Hugh Waddington,Mark Wilson,Beibei Yuan,John-Arne Røttingen

DOI: https://doi.org/10.1186/s13643-015-0080-9

2015-07-10

Abstract:Those planning, managing and working in health systems worldwide routinely need to make decisions regarding strategies to improve health care and promote equity. Systematic reviews of different kinds can be of great help to these decision-makers, providing actionable evidence at every step in the decision-making process. Although there is growing recognition of the importance of systematic reviews to inform both policy decisions and produce guidance for health systems, a number of important methodological and evidence uptake challenges remain and better coordination of existing initiatives is needed. The Alliance for Health Policy and Systems Research, housed within the World Health Organization, convened an Advisory Group on Health Systems Research (HSR) Synthesis to bring together different stakeholders interested in HSR synthesis and its use in decision-making processes. We describe the rationale of the Advisory Group and the six areas of its work and reflects on its role in advancing the field of HSR synthesis. We argue in favour of greater cross-institutional collaborations, as well as capacity strengthening in low- and middle-income countries, to advance the science and practice of health systems research synthesis. We advocate for the integration of quasi-experimental study designs in reviews of effectiveness of health systems intervention and reforms. The Advisory Group also recommends adopting priority-setting approaches for HSR synthesis and increasing the use of findings from systematic reviews in health policy and decision-making.
Evidence based policy making and the ‘art’ of commissioning – how English healthcare commissioners access and use information and academic research in ‘real life’ decision-making: an empirical qualitative study

Lesley Wye,Emer Brangan,Ailsa Cameron,John Gabbay,Jonathan H. Klein,Catherine Pope

DOI: https://doi.org/10.1186/s12913-015-1091-x

2015-06-01

BMC Health Services Research

Abstract:BackgroundPolicymakers such as English healthcare commissioners are encouraged to adopt ‘evidence-based policy-making’, with ‘evidence’ defined by researchers as academic research. To learn how academic research can influence policy, researchers need to know more about commissioning, commissioners’ information seeking behaviour and the role of research in their decisions.MethodsIn case studies of four commissioning organisations, we interviewed 52 people including clinical and managerial commissioners, observed 14 commissioning meetings and collected documentation e.g. meeting minutes and reports. Using constant comparison, data were coded, summarised and analysed to facilitate cross case comparison.ResultsThe ‘art of commissioning’ entails juggling competing agendas, priorities, power relationships, demands and personal inclinations to build a persuasive, compelling case. Policymakers sought information to identify options, navigate ways through, justify decisions and convince others to approve and/or follow the suggested course. ‘Evidence-based policy-making’ usually meant pragmatic selection of ‘evidence’ such as best practice guidance, clinicians’ and users’ views of services and innovations from elsewhere. Inconclusive or negative research was unhelpful in developing policymaking plans and did not inform disinvestment decisions. Information was exchanged through conversations and stories, which were fast, flexible and suited the rapidly changing world of policymaking. Local data often trumped national or research-based evidence. Local evaluations were more useful than academic research.DiscussionCommissioners are highly pragmatic and will only use information that helps them create a compelling case for action.Therefore, researchers need to start producing more useful information.ConclusionsTo influence policymakers’ decisions, researchers need to 1) learn more about local policymakers’ priorities 2) develop relationships of mutual benefit 3) use verbal instead of writtencommunication 4) work with intermediaries such as public health consultants and 5) co-produce local evaluations.

health care sciences & services
Rapid reviews and the methodological rigor of evidence synthesis: a JBI position statement

Andrea C. Tricco,Hanan Khalil,Cheryl Holly,Garumma Feyissa,Christina Godfrey,Catrin Evans,Diane Sawchuck,Morankar Sudhakar,Constantine Asahngwa,Daphne Stannard,Misra Abdulahi,Laura Bonnano,Edoardo Aromataris,Patricia McInerney,Rosemary Wilson,Dong Pang,Zhiwen Wang,Ana Filipa Cardoso,Micah D.J. Peters,Casey Marnie,Timothy Barker,Danielle Pollock,Alexa McArthur,Zachary Munn

DOI: https://doi.org/10.11124/jbies-21-00371

2022-02-09

JBI Evidence Synthesis

Abstract:ABSTRACT: The demand for rapid reviews has exploded in recent years. A rapid review is an approach to evidence synthesis that provides timely information to decision-makers (eg, health care planners, providers, policymakers, patients) by simplifying the evidence synthesis process. A rapid review is particularly appealing for urgent decisions. JBI is a world-renowned international collaboration for evidence synthesis and implementation methodologies. The principles for JBI evidence synthesis include comprehensiveness, rigor, transparency, and a focus on applicability to clinical practice. As such, JBI has not yet endorsed a specific approach for rapid reviews. In this paper, we compare rapid reviews versus other types of evidence synthesis, provide a range of rapid evidence products, outline how to appraise the quality of rapid reviews, and present the JBI position on rapid reviews. JBI Collaborating Centers conduct rapid reviews for decision-makers in specific circumstances, such as limited time or funding constraints. A standardized approach is not used for these cases;instead, the evidence synthesis methods are tailored to the needs of the decision-maker. The urgent need to deliver timely evidence to decision-makers poses challenges to JBI's mission to produce high-quality, trustworthy evidence. However, JBI recognizes the value of rapid reviews as part of the evidence synthesis ecosystem. As such, it is recommended that rapid reviews be conducted with the same methodological rigor and transparency expected of JBI reviews. Most importantly, transparency is essential, and the rapid review should clearly report where any simplification in the steps of the evidence synthesis process has been taken.

English Else
Framework for the synthesis of non-randomised studies and randomised controlled trials: a guidance on conducting a systematic review and meta-analysis for healthcare decision making

Grammati Sarri,Elisabetta Patorno,Hongbo Yuan,Jianfei Guo,Dimitri Bennett,Xuerong Wen,Andrew R Zullo,Joan Largent,Mary Panaccio,Mugdha Gokhale,Daniela Claudia Moga,M Sanni Ali,Thomas P A Debray,Jianfei (Jeff) Guo

DOI: https://doi.org/10.1136/bmjebm-2020-111493

2020-12-09

Abstract:Introduction: High-quality randomised controlled trials (RCTs) provide the most reliable evidence on the comparative efficacy of new medicines. However, non-randomised studies (NRS) are increasingly recognised as a source of insights into the real-world performance of novel therapeutic products, particularly when traditional RCTs are impractical or lack generalisability. This means there is a growing need for synthesising evidence from RCTs and NRS in healthcare decision making, particularly given recent developments such as innovative study designs, digital technologies and linked databases across countries. Crucially, however, no formal framework exists to guide the integration of these data types. Objectives and Methods: To address this gap, we used a mixed methods approach (review of existing guidance, methodological papers, Delphi survey) to develop guidance for researchers and healthcare decision-makers on when and how to best combine evidence from NRS and RCTs to improve transparency and build confidence in the resulting summary effect estimates. Results: Our framework comprises seven steps on guiding the integration and interpretation of evidence from NRS and RCTs and we offer recommendations on the most appropriate statistical approaches based on three main analytical scenarios in healthcare decision making (specifically, ‘high-bar evidence’ when RCTs are the preferred source of evidence, ‘medium,’ and ‘low’ when NRS is the main source of inference). Conclusion: Our framework augments existing guidance on assessing the quality of NRS and their compatibility with RCTs for evidence synthesis, while also highlighting potential challenges in implementing it. This manuscript received endorsement from the International Society for Pharmacoepidemiology.
Living systematic reviews: A novel mechanism for improving efficiency and quality of evidence synthesis in oncology.

I. Riaz,R. Siddiqi,N. Asghar,Elizabeth Jane Cathcart-Rake,V. Herasevich,V. Montori,Zhen Wang,Ronald S. Go,S. V. Rajkumar,M. H. Murad

DOI: https://doi.org/10.1200/jco.2019.37.27_suppl.241

IF: 45.3

2019-09-20

Journal of Clinical Oncology

Abstract:241 Background: In a rapidly moving field, such as cancer immunotherapy, where immune checkpoint inhibitors (ICIs) are used across 14 different tumor types, patients may receive suboptimal treatment or even be harmed if information on toxicity is not readily translated for use in clinical practice. Every single systematic review and meta-analysis which attempted to summarize toxicity of immune checkpoint inhibitors (ICIs) quickly became outdated. A living systematic review, which is defined as a systematic review that is continually updated to incorporate relevant new evidence as it becomes available, is necessary in this situation. Methods: The process of creating a living systematic review started with the creation of a comprehensive search designed by a librarian experienced in systematic reviews in collaboration with the study’s principle investigator. Search was constantly updated every 3 months and evidence is synthesized in a series of steps (microtasks) using a combination of human and augmented intelligence. A complete infrastructure is being developed and it includes automated cumulative meta-analysis and an online reporting platform which will constantly update information for clinicians and patients in a live manner. Results: We screened 6746 studies during Sep 2018-March 2019 and identified 6746 studies and we were able to successfully maintain up-to-date toxicity estimates for immune mediated adverse events over this period while maintaining the rigor of a conventional systematic review. Eventually, we will integrate the steps of LSR into one, user-friendly, semi-automated format which can independently provide accurate estimates and feed into and support a living guidelines platform through shared Application Programing Interface (APIs). Conclusions: LSRs are feasible, efficient, and when fully developed can reduce redundancy and waste in medical research, improve the quality of evidence, reduce human effort and support living and dynamic guidelines to facilitate truly informed shared decision making.

Medicine
Innovations in the systematic review of text and opinion

Alexa McArthur,Jitka Klugárová,Hu Yan,Silvia Florescu,McArthur, Alexa,Florescu, Silvia,Klugárová, Jitka,Yan, Hu

DOI: https://doi.org/10.1097/XEB.0000000000000060

2019-10-03

Abstract:ABSTRACT Background: Evidence-based healthcare focuses on the need to use interventions that are supported by the best available and most up-to-date evidence or knowledge. Many clinical questions cannot be fully answered by evidence derived from quantitative or qualitative research designs alone, since many areas in healthcare are supported by clinicians’ tacit knowledge derived from their clinical experiences. In this situation, evidence generated from a systematic review of text and opinion may be required as the best available evidence. The aim of this study is to highlight the importance and role of expert opinion synthesis in healthcare, and present results of an international methodological group review. Methods: A methodological group was formed to review this approach, and update the guidance and processes for undertaking a systematic review of text and opinion. Results: The methodology of systematic reviews of text and opinion had already been developed by the Joanna Briggs Institute. We reviewed and updated several steps in the process, such as inclusion criteria, search strategy, critical appraisal and data extraction. Conclusions: The Joanna Briggs Institute methodology for the systematic review of text and opinion is unique, and continuously evolving and being further developed. Systematic reviews of text and opinion may be considered as legitimate sources of evidence, especially when there is an absence of other research designs.

Decision makers need constantly updated evidence synthesis

Need for evidence synthesis for quality control of healthcare decision-making

Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap

Living systematic review: 1. Introduction—the why, what, when, and how

Flexible synthesis to deliver relevant evidence for decision-making

Future of evidence ecosystem series: 2. current opportunities and need for better tools and methods

A practical guide to living evidence: Reducing the knowledge-to-practice gap

A fresh approach to evidence synthesis

A narrative review of recent tools and innovations toward automating living systematic reviews and evidence syntheses

A living critical interpretive synthesis to yield a framework on the production and dissemination of living evidence syntheses for decision-making

SHOW ME the evidence: Features of an approach to reliably deliver research evidence to those who need it

ALL-IN meta-analysis: breathing life into living systematic reviews

Appraised evidence and decision making in clinical practice.

Living systematic reviews: 2. Combining human and machine effort

Methods proposed for monitoring the implementation of evidence-based research: a cross-sectional study

Advancing the field of health systems research synthesis

Evidence based policy making and the ‘art’ of commissioning – how English healthcare commissioners access and use information and academic research in ‘real life’ decision-making: an empirical qualitative study

Rapid reviews and the methodological rigor of evidence synthesis: a JBI position statement

Framework for the synthesis of non-randomised studies and randomised controlled trials: a guidance on conducting a systematic review and meta-analysis for healthcare decision making

Living systematic reviews: A novel mechanism for improving efficiency and quality of evidence synthesis in oncology.

Innovations in the systematic review of text and opinion