Scientists are building a giant 'evidence banks' to create policies that actually work

Helen Pearson

DOI: https://doi.org/10.1038/d41586-024-03100-2

IF: 64.8

2024-09-22

Nature

Abstract:Funders launch projects with US 70-million to develop tools that make rapid syntheses of the world's science.

multidisciplinary sciences

What problem does this paper attempt to address?

What's the best way to tackle climate change? An 'evidence bank' could help scientists find answers

Helen Pearson

DOI: https://doi.org/10.1038/d41586-024-01683-4

IF: 64.8

2024-06-07

Nature

Abstract:Sythensizing research on which policies are most effective is a key priority in climate science, advocates say. Sythensizing research on which policies are most effective is a key priority in climate science, advocates say.

multidisciplinary sciences
Science-policy advisers shape programmes that solve real-world problems

Amy DePaul

DOI: https://doi.org/10.1038/d41586-024-03040-x

IF: 64.8

2024-09-19

Nature

Abstract:Four scientists discuss how they broke into the field and how they use evidence to tackle society's problems. Four scientists discuss how they broke into the field and how they use evidence to tackle society's problems.

multidisciplinary sciences
SHOW ME the evidence: Features of an approach to reliably deliver research evidence to those who need it

John N Lavis,Jeremy M Grimshaw,Ruth Stewart,Julian Elliott,Will Moy,Joerg J Meerpohl

DOI: https://doi.org/10.1002/14651858.ed000170

IF: 8.4

2024-11-22

Cochrane Database of Systematic Reviews

Abstract:The world is poised for a step‐change improvement in how we use evidence to address societal challenges. Given the speed at which plans are being made to support this once‐in‐a‐generation transformation, the Implementation Council of the Global Commission on Evidence to Address Societal Challenges developed a working version of the features of an approach to reliably getting research evidence to those who need it and achieved consensus among leaders from the Implementation Council, as well as the Alliance for Living Evidence (Alive) Council and Evidence Synthesis International (ESI). Drawing an acronym from the first letter of each of the six features, the 'SHOW ME the evidence' features are as follows. The 100+ contributing authors from across the 'evidence synthesis and support' world want to ensure that our future plans are firmly rooted in an agreed‐upon summary of all we have learned together over these past four or so years, and to signal a mutual accountability among many of the key players involved in providing evidence support that we will each do our part in delivering on the promise that motivates these plans. Given that much of the momentum for transformation is currently focused on living evidence syntheses and the infrastructure needed to support them, we give this form of evidence disproportionate focus here. An even more diverse set of partners should be engaged in designing and executing an inclusive process for the refinement or even reshaping of these features over time, as well as their ongoing operationalization. This includes more types of decision makers, those working with more forms of evidence, and funders, as well as even more contributors from across the Global South. Support systems locally that use many forms of research evidence to help address local priorities Every jurisdiction needs a reliable evidence‐support system to get whatever forms of evidence are needed to address a local priority into the hands of those who need it, when they need it, in whatever form they need it, and with any required caveats about its currency, quality, and local applicability [1]. Locally can mean nations as well as subnational jurisdictions like provinces and cities. It can mean formal regional groupings of countries like the European Union and informal regional groupings of small countries with shared challenges. It can also mean systems, like the health or social‐care system. The forms of evidence can include research evidence from the 'local' context (e.g. data analytics, evaluation, and behavioural or implementation research), research evidence from around the world (i.e. evidence synthesis), and other types of information (e.g. horizon scanning and people's lived experiences) and ways of knowing (e.g. Indigenous knowledge). Addressing a local priority is ideally informed by an understanding of a problem (and its causes and alternative ways of framing it), options to address the problem (including those already in use at a small scale), implementation considerations, and how to monitor implementation and evaluate impact. Research evidence can inform such understandings alongside political and social insights. Those who need research evidence can include government policy makers (from central agencies like Treasury, line departments like Education, and legislatures), organizational leaders (from both nongovernmental organizations and private companies), professionals (like nurses, teachers and veterinarians), and citizens (in the broadest sense of that term, and inclusive of undocumented individuals, as described in section 3.6 of the Global Evidence Commission report 2022). They also need enablers, culture and capacity for evidence use. Many decision makers need actionable insights from research evidence quickly when a 'window of opportunity' opens. Sometimes these windows are open for days, other times weeks, and rarely for longer. Evidence support can now work at the same speed as decision‐making processes. Some decision makers may want the evidence presented to them as 'best buys' (e.g. Global Education Evidence Advisory Panel), others by broad approach (e.g. Education Endowment Foundation), and still others by branded programme (e.g. IES What Works Clearinghouse). Applicability can mean both for local contexts and for groups in a range of contexts, including groups most affected by historical and acute inequities. Harmonized efforts globally that make it easier to learn from others around the world One aspect of evidence support that can now best be undertaken through harmonized efforts globally is to provide regularly updated summaries of what we have learned from around the world and how these findings vary by groups and contexts. 'Living evidence synthesis' is a relatively new approach to producing and maintaining these summaries [2]. The take‐up of this approach accelerated during the COVID‐19 pandemic -Abstract Truncated-

medicine, general & internal
Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy?

Paul Cairney,Kathryn Oliver

DOI: https://doi.org/10.1186/s12961-017-0192-x

2017-04-26

Health Research Policy and Systems

Abstract:There is extensive health and public health literature on the ‘evidence-policy gap’, exploring the frustrating experiences of scientists trying to secure a response to the problems and solutions they raise and identifying the need for better evidence to reduce policymaker uncertainty. We offer a new perspective by using policy theory to propose research with greater impact, identifying the need to use persuasion to reduce ambiguity, and to adapt to multi-level policymaking systems.We identify insights from secondary data, namely systematic reviews, critical analysis and policy theories relevant to evidence-based policymaking. The studies are drawn primarily from countries such as the United States, United Kingdom, Canada, Australia and New Zealand. We combine empirical and normative elements to identify the ways in which scientists can, do and could influence policy.We identify two important dilemmas, for scientists and researchers, that arise from our initial advice. First, effective actors combine evidence with manipulative emotional appeals to influence the policy agenda – should scientists do the same, or would the reputational costs outweigh the policy benefits? Second, when adapting to multi-level policymaking, should scientists prioritise ‘evidence-based’ policymaking above other factors? The latter includes governance principles such the ‘co-production’ of policy between local public bodies, interest groups and service users. This process may be based primarily on values and involve actors with no commitment to a hierarchy of evidence.We conclude that successful engagement in ‘evidence-based policymaking’ requires pragmatism, combining scientific evidence with governance principles, and persuasion to translate complex evidence into simple stories. To maximise the use of scientific evidence in health and public health policy, researchers should recognise the tendency of policymakers to base judgements on their beliefs, and shortcuts based on their emotions and familiarity with information; learn ‘where the action is’, and be prepared to engage in long-term strategies to be able to influence policy; and, in both cases, decide how far you are willing to go to persuade policymakers to act and secure a hierarchy of evidence underpinning policy. These are value-driven and political, not just ‘evidence-based’, choices.

health policy & services
Optimally generate policy-based evidence before scaling

John A. List

DOI: https://doi.org/10.1038/s41586-023-06972-y

IF: 64.8

2024-02-15

Nature

Abstract:Social scientists have increasingly turned to the experimental method to understand human behaviour. One critical issue that makes solving social problems difficult is scaling up the idea from a small group to a larger group in more diverse situations. The urgency of scaling policies impacts us every day, whether it is protecting the health and safety of a community or enhancing the opportunities of future generations. Yet, a common result is that, when we scale up ideas, most experience a 'voltage drop'—that is, on scaling, the cost–benefit profile depreciates considerably. Here I argue that, to reduce voltage drops, we must optimally generate policy-based evidence. Optimality requires answering two crucial questions: what information should be generated and in what sequence. The economics underlying the science of scaling provides insights into these questions, which are in some cases at odds with conventional approaches. For example, there are important situations in which I advocate flipping the traditional social science research model to an approach that, from the beginning, produces the type of policy-based evidence that the science of scaling demands. To do so, I propose augmenting efficacy trials by including relevant tests of scale in the original discovery process, which forces the scientist to naturally start with a recognition of the big picture: what information do I need to have scaling confidence?

multidisciplinary sciences
NSF bets big on turning research into prosperity

Jeffrey Mervis

DOI: https://doi.org/10.1126/science.ado3857

IF: 56.9

2024-02-02

Science

Abstract:Agency funds first round of 10 "innovation engines" designed to spur regional economic growth

multidisciplinary sciences
Leveraging AI to improve evidence synthesis in conservation

Oded Berger-Tal,Bob B.M. Wong,Carrie Ann Adams,Daniel T. Blumstein,Ulrika Candolin,Matthew J. Gibson,Alison L. Greggor,Malgorzata Lagisz,Biljana Macura,Catherine J. Price,Breanna J. Putman,Lysanne Snijders,Shinichi Nakagawa

DOI: https://doi.org/10.1016/j.tree.2024.04.007

IF: 20.589

2024-06-06

Trends in Ecology & Evolution

Abstract:Systematic evidence syntheses (systematic reviews and maps) summarize knowledge and are used to support decisions and policies in a variety of applied fields, from medicine and public health to biodiversity conservation. However, conducting these exercises in conservation is often expensive and slow, which can impede their use and hamper progress in addressing the current biodiversity crisis. With the explosive growth of large language models (LLMs) and other forms of artificial intelligence (AI), we discuss here the promise and perils associated with their use. We conclude that, when judiciously used, AI has the potential to speed up and hopefully improve the process of evidence synthesis, which can be particularly useful for underfunded applied fields, such as conservation science.

genetics & heredity,ecology,evolutionary biology
Unlocking the power of global collaboration: building a stronger evidence ecosystem together

Zoe Jordan,Vivian Welch,Karla Soares-Weiser

DOI: https://doi.org/10.1002/14651858.ed000166

IF: 8.4

2024-04-17

Cochrane Database of Systematic Reviews

Abstract:Across the global evidence ecosystem, numerous organizations share a common vision and mission: to promote evidence‐based decision making worldwide. These organizations, including JBI, the Cochrane Collaboration, and the Campbell Collaboration, have each made an indelible imprint on the evidence‐based movement and have been identified as, "a crucial mechanism to facilitate the synthesis, transfer, and implementation of evidence into health care policy and practice."[[1], p.211] While the benefits of global collaboration have been well established for some time, achieving impact at scale will require a fundamental shift in mindset. The COVID‐19 pandemic marked a turning point for evidence‐based health care and decision making. It provided a unique context whereby policymakers, healthcare providers, researchers, and the public required immediate access to trustworthy evidence to make decisions. We collectively faced major challenges in translating a rapidly evolving body of new evidence into tangible response efforts, with health policy decisions receiving unprecedented public attention. The 'stress test' of COVID‐19, and the many postpandemic initiatives that followed, highlighted the need for more effective strategies, institutional mechanisms, and capacities to systematically mobilize and contextualize the best available evidence for rapid decision‐making for effective and equitable public health responses.[2, 3, 4] Each of our organizations responded to COVID‐19 in different ways and were able to provide access to reliable evidence. Yet it is essential to acknowledge the challenges of sustaining funding, upholding methodological rigour, and ensuring diversity and inclusivity in our collective endeavours. Our demonstrated success in enhancing global health care, education, and social policy underscores the value of collaborative, evidence‐based approaches in addressing the world's most pressing challenges. We find ourselves at a unique juncture where our respective global collaborative evidence networks (JBI, Cochrane, and Campbell) must reimagine the way we work together to facilitate and engage in multidisciplinary, transdisciplinary, and interdisciplinary research, dissemination, knowledge sharing and knowledge translation to generate impact at scale across the evidence ecosystem. It is time to develop interagency collaboration as a coherent programme rather than a series of standalone efforts. There is significant potential in our ability to orchestrate, integrate, co‐ordinate, and align our activities to identify opportunities for mutual benefit, learning, and impact. A call to action . One of the most significant benefits of our respective global networks is our capacity to transcend geographical boundaries. By facilitating better global interagency collaboration, we enable the pooling of expertise and knowledge in the field of evidence‐based practice, and the result is a more holistic and nuanced understanding of complex issues, leading to improved decision making at both local and global levels. Examples of this may include much deeper collaboration on methodologies and standards for synthesis that reflect the diversity of evidence to respond to global challenges; a more co‐ordinated approach to the prioritization of synthesis efforts to avoid duplication of effort; and better, more meaningful partnership on the contextualization or localization of evidence for policy and practice. Going forward, it is incumbent upon us to support and strengthen our networks and the relationships between them, recognizing the invaluable contributions we can collectively make to improving the well‐being of individuals and communities around the world. The path to a brighter, more evidence‐based future lies in continued collaboration and our unwavering commitment to the delivery of trustworthy evidence. Collaboration across our global networks, not just within them, is now not merely a choice but a necessity in our increasingly interconnected world.

medicine, general & internal
How to bring research evidence into policy? Synthesizing strategies of five research projects in low-and middle-income countries

Séverine Erismann,Maria Amalia Pesantes,David Beran,Andrea Leuenberger,Andrea Farnham,Monica Berger Gonzalez de White,Niklaus Daniel Labhardt,Fabrizio Tediosi,Patricia Akweongo,August Kuwawenaruwa,Jakob Zinsstag,Fritz Brugger,Claire Somerville,Kaspar Wyss,Helen Prytherch

DOI: https://doi.org/10.1186/s12961-020-00646-1

2021-03-06

Health Research Policy and Systems

Abstract:Abstract Background Addressing the uptake of research findings into policy-making is increasingly important for researchers who ultimately seek to contribute to improved health outcomes. The aims of the Swiss Programme for Research on Global Issues for Development (r4d Programme) initiated by the Swiss National Science Foundation and the Swiss Agency for Development and Cooperation are to create and disseminate knowledge that supports policy changes in the context of the 2030 Agenda for Sustainable Development. This paper reports on five r4d research projects and shows how researchers engage with various stakeholders, including policy-makers, in order to assure uptake of the research results. Methods Eleven in-depth interviews were conducted with principal investigators and their research partners from five r4d projects, using a semi-structured interview guide. The interviews explored the process of how stakeholders and policy-makers were engaged in the research project. Results Three key strategies were identified as fostering research uptake into policies and practices: (S1) stakeholders directly engaged with and sought evidence from researchers; (S2) stakeholders were involved in the design and throughout the implementation of the research project; and (S3) stakeholders engaged in participatory and transdisciplinary research approaches to coproduce knowledge and inform policy. In the first strategy, research evidence was directly taken up by international stakeholders as they were actively seeking new evidence on a very specific topic to up-date international guidelines. In the second strategy, examples from two r4d projects show that collaboration with stakeholders from early on in the projects increased the likelihood of translating research into policy, but that the latter was more effective in a supportive and stable policy environment. The third strategy adopted by two other r4d projects demonstrates the benefits of promoting colearning as a way to address potential power dynamics and working effectively across the local policy landscape through robust research partnerships. Conclusions This paper provides insights into the different strategies that facilitate collaboration and communication between stakeholders, including policy-makers, and researchers. However, it remains necessary to increase our understanding of the interests and motivations of the different actors involved in the process of influencing policy, identify clear policy-influencing objectives and provide more institutional support to engage in this complex and time-intensive process.

health policy & services
Enhancing environmental policy through evidence synthesis: a review of the Environmental Evidence for the Future (EEF) Initiative

Kathryn Anne Monk

DOI: https://doi.org/10.1186/s13750-024-00329-2

2024-03-31

Environmental Evidence

Abstract:The Environmental Evidence for the Future (EEF) Initiative emerged in response to the challenges and opportunities presented by the UK's decision to leave the European Union and its associated Environmental Frameworks. The Natural Environment Research Council (NERC), working closely with the Collaboration for Environmental Evidence (CEE) and UK stakeholders, developed the initiative to identify and address crucial evidence gaps, offering a long-term vision for environmental policy and sustainability. The EEF Initiative progressed through three stages: strategic priority identification, NERC panel award selection, and the production of Systematic Maps of existing evidence. The first stage involved collaborative workshops across the UK to identify key knowledge gaps in environmental science. The subsequent prioritisation resulted in 65 challenges across 10 thematic areas. The second stage saw NERC initiating, with CEE support, an open call for research proposals emphasising the use of evidence synthesis methodology. The selection process, balancing topic importance and applicant expertise, led to funding for five projects. The final stage involved the production of Systematic Maps of existing evidence based on the CEE Guidelines and Standards, providing a structured overview of existing literature on specific topics. The EEF Initiative demonstrated effective collaboration between UKRI (NERC), an independent non-profit (CEE), academia, and government agencies, addressing critical environmental challenges through rigorous evidence synthesis methodologies. The programme enhanced understanding and utilisation of these methodologies within the research community. Key lessons include the importance of inclusive priority-setting, differentiation between broad policy questions and specific Systematic Map questions, recognition of the value of Systematic Maps, and the role of experience in evidence synthesis teams. As policymakers and researchers navigate environmental policies in a resource-constrained environment, the EEF Initiative highlights the cost-effectiveness and efficiency of systematic mapping and review processes for evidence-based decision-making. The success of funding through NERC sets a precedent for future thematic evidence focused programmes, emphasising the need for continued support in developing synthesis skills among researchers and encouraging direct government commissions for targeted and responsive evidence. The EEF Initiative serves as a model for effective collaboration, providing valuable insights into addressing evidence gaps and shaping evidence-based policymaking in an ever-evolving environmental landscape.

environmental sciences
The new evidence-based policy: public participation between ‘hard evidence’ and democracy in practice

Helen Pallett

DOI: https://doi.org/10.1332/174426419x15704985880872

2020-05-01

Evidence and Policy: A Journal of Research, Debate and Practice

Abstract:Background: Debates about evidence-based policy (EBP) were revived in the UK in the 2010s in the context of civil service reform and changing practices of policy making, including institutionalisation of public participation in science policy making. Aims and objectives: This paper aims to explore this revival of interest in EBP in the context of the Government-funded public participation programme Sciencewise, which supports and promotes public dialogues in science policy making. It is based on in-depth ethnographic study of the programme during 2013, considering the impacts on Sciencewise practices and working understandings of engaging in the EBP debate. There is a particular focus on the advantages and disadvantages of categorising public participation as a source of evidence-based policy as opposed to presenting participation as a democratic act which is separate from discussions of EBP. Key conclusions: At different times Sciencewise actors moved between these stances in order to gain credibility and attention for their work, and to situate the outcomes of public participation processes in a broader policy context. In some instances the presentation of outputs from public participation processes as legitimate evidence for policy gave them greater influence and enriched broader discussions about the meaning and practice of open policy. However, it also frequently led to their dismissal on methodological grounds, inhibiting serious engagement with their outputs and challenging internal frameworks for evaluation and learning.
Detecting signal from science:The structure of research communities and prior knowledge improves prediction of genetic regulatory experiments

Alexander V. Belikov,Andrey Rzhetsky,James Evans

DOI: https://doi.org/10.48550/arXiv.2008.09985

2020-08-23

Social and Information Networks

Abstract:The explosive growth of scientists, scientific journals, articles and findings in recent years exponentially increases the difficulty scientists face in navigating prior knowledge. This challenge is exacerbated by uncertainty about the reproducibility of published findings. The availability of massive digital archives, machine reading and extraction tools on the one hand, and automated high-throughput experiments on the other, allow us to evaluate these challenges at scale and identify novel opportunities for accelerating scientific advance. Here we demonstrate a Bayesian calculus that enables the positive prediction of robust, replicable scientific claims with findings automatically extracted from published literature on gene interactions. We matched these findings, filtered by science, with unfiltered gene interactions measured by the massive LINCS L1000 high-throughput experiment to identify and counteract sources of bias. Our calculus is built on easily extracted publication meta-data regarding the position of a scientific claim within the web of prior knowledge, and its breadth of support across institutions, authors and communities, revealing that scientifically focused but socially and institutionally independent research activity is most likely to replicate. These findings recommend policies that go against the common practice of channeling biomedical research funding into centralized research consortia and institutes rather than dispersing it more broadly. Our results demonstrate that robust scientific findings hinge upon a delicate balance of shared focus and independence, and that this complex pattern can be computationally exploited to decode bias and predict the replicability of published findings. These insights provide guidance for scientists navigating the research literature and for science funders seeking to improve it.
Science could solve some of the world's biggest problems. Why aren't governments using it?

Helen Pearson

DOI: https://doi.org/10.1038/d41586-024-03906-0

IF: 64.8

2024-12-04

Nature

Abstract:A Nature global survey finds that most specialists are unhappy with systems to provide science advice to policymakers. A Nature global survey finds that most specialists are unhappy with systems to provide science advice to policymakers.

multidisciplinary sciences
Does big data serve policy? Not without context. An experiment with in silico social science

Chris Graziul,Alexander Belikov,Ishanu Chattopadyay,Ziwen Chen,Hongbo Fang,Anuraag Girdhar,Xiaoshuang Jia,P M Krafft,Max Kleiman-Weiner,Candice Lewis,Chen Liang,John Muchovej,Alejandro Vientós,Meg Young,James Evans

DOI: https://doi.org/10.1007/s10588-022-09362-3

Abstract:The DARPA Ground Truth project sought to evaluate social science by constructing four varied simulated social worlds with hidden causality and unleashed teams of scientists to collect data, discover their causal structure, predict their future, and prescribe policies to create desired outcomes. This large-scale, long-term experiment of in silico social science, about which the ground truth of simulated worlds was known, but not by us, reveals the limits of contemporary quantitative social science methodology. First, problem solving without a shared ontology-in which many world characteristics remain existentially uncertain-poses strong limits to quantitative analysis even when scientists share a common task, and suggests how they could become insurmountable without it. Second, data labels biased the associations our analysts made and assumptions they employed, often away from the simulated causal processes those labels signified, suggesting limits on the degree to which analytic concepts developed in one domain may port to others. Third, the current standard for computational social science publication is a demonstration of novel causes, but this limits the relevance of models to solve problems and propose policies that benefit from the simpler and less surprising answers associated with most important causes, or the combination of all causes. Fourth, most singular quantitative methods applied on their own did not help to solve most analytical challenges, and we explored a range of established and emerging methods, including probabilistic programming, deep neural networks, systems of predictive probabilistic finite state machines, and more to achieve plausible solutions. However, despite these limitations common to the current practice of computational social science, we find on the positive side that even imperfect knowledge can be sufficient to identify robust prediction if a more pluralistic approach is applied. Applying competing approaches by distinct subteams, including at one point the vast TopCoder.com global community of problem solvers, enabled discovery of many aspects of the relevant structure underlying worlds that singular methods could not. Together, these lessons suggest how different a policy-oriented computational social science would be than the computational social science we have inherited. Computational social science that serves policy would need to endure more failure, sustain more diversity, maintain more uncertainty, and allow for more complexity than current institutions support.
Theoretical and technological building blocks for an innovation accelerator

F. van Harmelen,G. Kampis,K. Börner,P. van den Besselaar,E. Schultes,C. Goble,P. Groth,B. Mons,S. Anderson,S. Decker,C. Hayes,T. Buecheler,D. Helbing

DOI: https://doi.org/10.1140/epjst/e2012-01692-1

2012-11-01

The European Physical Journal Special Topics

Abstract:AbstractModern science is a main driver of technological innovation. The efficiency of the scientific system is of key importance to ensure the competitiveness of a nation or region. However, the scientific system that we use today was devised centuries ago and is inadequate for our current ICT-based society: the peer review system encourages conservatism, journal publications are monolithic and slow, data is often not available to other scientists, and the independent validation of results is limited. The resulting scientific process is hence slow and sloppy. Building on the Innovation Accelerator paper by Helbing and Balietti [1], this paper takes the initial global vision and reviews the theoretical and technological building blocks that can be used for implementing an innovation (in first place: science) accelerator platform driven by re-imagining the science system. The envisioned platform would rest on four pillars: (i) Redesign the incentive scheme to reduce behavior such as conservatism, herding and hyping; (ii) Advance scientific publications by breaking up the monolithic paper unit and introducing other building blocks such as data, tools, experiment workflows, resources; (iii) Use machine readable semantics for publications, debate structures, provenance etc. in order to include the computer as a partner in the scientific process, and (iv) Build an online platform for collaboration, including a network of trust and reputation among the different types of stakeholders in the scientific system: scientists, educators, funding agencies, policy makers, students and industrial innovators among others. Any such improvements to the scientific system must support the entire scientific process (unlike current tools that chop up the scientific process into disconnected pieces), must facilitate and encourage collaboration and interdisciplinarity (again unlike current tools), must facilitate the inclusion of intelligent computing in the scientific process, must facilitate not only the core scientific process, but also accommodate other stakeholders such science policy makers, industrial innovators, and the general public. We first describe the current state of the scientific system together with up to a dozen new key initiatives, including an analysis of the role of science as an innovation accelerator. Our brief survey will show that there exist many separate ideas and concepts and diverse stand-alone demonstrator systems for different components of the ecosystem with many parts are still unexplored, and overall integration lacking. By analyzing a matrix of stakeholders vs. functionalities, we identify the required innovations. We (non-exhaustively) discuss a few of them: Publications that are meaningful to machines, innovative reviewing processes, data publication, workflow archiving and reuse, alternative impact metrics, tools for the detection of trends, community formation and emergence, as well as modular publications, citation objects and debate graphs. To summarize, the core idea behind the Innovation Accelerator is to develop new incentive models, rules, and interaction mechanisms to stimulate true innovation, revolutionizing the way in which we create knowledge and disseminate information.Graphical abstract
Large-scale research infrastructure projects: A conceptual review for science policy and management

David C Eggleton

DOI: https://doi.org/10.1177/00368504241266555

IF: 1.512

2024-12-06

Science Progress

Abstract:Science Progress, Volume 107, Issue 4, October-December 2024. This article reviews 'large-scale research infrastructures' work and other relevant literature from the science policy and management domains. Through a systematic literature review, the study identifies that there are no firm inclusion or exclusion criteria for a large-scale research infrastructure. The findings identify the need for filling this knowledge gap to support future analyses for large-scale research infrastructures to help scientists and science policymakers understand, plan, and evaluate their own work. A refined version of one of the concepts examined in this article, the 'large international science project', provides the most fruitful starting point.

multidisciplinary sciences
Combining support for hypotheses over heterogeneous studies with Bayesian Evidence Synthesis: A simulation study

Thom Benjamin Volker,Irene Klugkist

DOI: https://doi.org/10.48550/arXiv.2312.15032

2023-12-23

Abstract:Scientific claims gain credibility by replicability, especially if replication under different circumstances and varying designs yields equivalent results. Aggregating results over multiple studies is, however, not straightforward, and when the heterogeneity between studies increases, conventional methods such as (Bayesian) meta-analysis and Bayesian sequential updating become infeasible. *Bayesian Evidence Synthesis*, built upon the foundations of the Bayes factor, allows to aggregate support for conceptually similar hypotheses over studies, regardless of methodological differences. We assess the performance of Bayesian Evidence Synthesis over multiple effect and sample sizes, with a broad set of (inequality-constrained) hypotheses using Monte Carlo simulations, focusing explicitly on the complexity of the hypotheses under consideration. The simulations show that this method can evaluate complex (informative) hypotheses regardless of methodological differences between studies, and performs adequately if the set of studies considered has sufficient statistical power. Additionally, we pinpoint challenging conditions that can lead to unsatisfactory results, and provide suggestions on handling these situations. Ultimately, we show that Bayesian Evidence Synthesis is a promising tool that can be used when traditional research synthesis methods are not applicable due to insurmountable between-study heterogeneity.

Methodology
Evidence-informed climate policy: mobilising strategic research and pooling expertise for rapid evidence generation

Anita Wreford,Suzanne Peace,Mark Reed,Justyna Bandola-Gill,Ragne Low,Andrew Cross

DOI: https://doi.org/10.1007/s10584-019-02483-w

2019-07-31

Climatic Change

Abstract:ClimateXChange (CXC) was established in 2011 as Scotland's Centre of Expertise on Climate Change. It aims to link research and policy, and to be a focal point the Scottish Government can call on for advice on climate change science, mitigation and adaptation actions, and analysis. Bringing together 15 Scottish institutions, CXC is an innovative organisation bridging the science policy gap. We outline CXC's formation and structure, and use CXC's experiences to date to highlight features that have been successful in facilitating knowledge exchange as well as on-going challenges. Based on our reflections of CXC, we demonstrate how boundary organisations can (i) meet near-term policy demand for evidence via small-scale, rapid response projects; and (ii) pool expertise from across the research community; whilst (iii) benefiting from longer-term, strategic programmes of research tackling complex, global challenges. We illustrate that while CXC has some unique characteristics, many of the lessons and experiences are applicable to other organisations seeking a greater connection between science and policy.

environmental sciences,meteorology & atmospheric sciences
We must discuss research environments

John Gallacher,Chris Webster

DOI: https://doi.org/10.1098/rsos.231742

IF: 3.5

2024-06-26

Royal Society Open Science

Abstract:A major challenge facing the biomedical community is creating and sustaining high-quality research environments. A literature search identified five common themes underlying biomedical research environments comprising collaboration, data access, user-led innovation, data provenance and a deep commitment to public and scientific benefit. Club theory is used to develop a model describing social structures that underpin these themes. It is argued that collaboration underlies impactful science and that collaboration is hindered by high transaction costs. This, combined with poorly defined property rights surrounding publicly funded data, limits the ability of data markets to operate efficiently. Although the science community is best placed to provide solutions for these issues, incentivization by funding agencies to increase the benefits of collaboration and reduce uncoordinated activity will be an accelerator. Given the complexity of emerging datasets and the collaborations needed to exploit them, trust-by-design solutions are suggested. The underlying motivational ‘glue’ that holds this activity together is the aesthetic and ethical value base underlying good science. The model has implications for data-driven science more generally. As biomedical science in the Global South develops, there is an opportunity to address foundational structural issues prospectively rather than inherit unwanted constraints of current practice.

multidisciplinary sciences
Developing and refining the methods for a 'one-stop shop' for research evidence about health systems

John N Lavis,Michael G Wilson,Kaelan A Moat,Amanda C Hammill,Jennifer A Boyko,Jeremy M Grimshaw,Signe Flottorp

DOI: https://doi.org/10.1186/1478-4505-13-10

2015-02-25

Abstract:Background: Policymakers, stakeholders and researchers have not been able to find research evidence about health systems using an easily understood taxonomy of topics, know when they have conducted a comprehensive search of the many types of research evidence relevant to them, or rapidly identify decision-relevant information in their search results. Methods: To address these gaps, we developed an approach to building a 'one-stop shop' for research evidence about health systems. We developed a taxonomy of health system topics and iteratively refined it by drawing on existing categorization schemes and by using it to categorize progressively larger bundles of research evidence. We identified systematic reviews, systematic review protocols, and review-derived products through searches of Medline, hand searches of several databases indexing systematic reviews, hand searches of journals, and continuous scanning of listservs and websites. We developed an approach to providing 'added value' to existing content (e.g., coding systematic reviews according to the countries in which included studies were conducted) and to expanding the types of evidence eligible for inclusion (e.g., economic evaluations and health system descriptions). Lastly, we developed an approach to continuously updating the online one-stop shop in seven supported languages. Results: The taxonomy is organized by governance, financial, and delivery arrangements and by implementation strategies. The 'one-stop shop', called Health Systems Evidence, contains a comprehensive inventory of evidence briefs, overviews of systematic reviews, systematic reviews, systematic review protocols, registered systematic review titles, economic evaluations and costing studies, health reform descriptions and health system descriptions, and many types of added-value coding. It is continuously updated and new content is regularly translated into Arabic, Chinese, English, French, Portuguese, Russian, and Spanish. Conclusions: Policymakers and stakeholders can now easily access and use a wide variety of types of research evidence about health systems to inform decision-making and advocacy. Researchers and research funding agencies can use Health Systems Evidence to identify gaps in the current stock of research evidence and domains that could benefit from primary research, systematic reviews, and review overviews.

Scientists are building a giant 'evidence banks' to create policies that actually work

What's the best way to tackle climate change? An 'evidence bank' could help scientists find answers

Science-policy advisers shape programmes that solve real-world problems

SHOW ME the evidence: Features of an approach to reliably deliver research evidence to those who need it

Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy?

Optimally generate policy-based evidence before scaling

NSF bets big on turning research into prosperity

Leveraging AI to improve evidence synthesis in conservation

Unlocking the power of global collaboration: building a stronger evidence ecosystem together

How to bring research evidence into policy? Synthesizing strategies of five research projects in low-and middle-income countries

Enhancing environmental policy through evidence synthesis: a review of the Environmental Evidence for the Future (EEF) Initiative

The new evidence-based policy: public participation between ‘hard evidence’ and democracy in practice

Detecting signal from science:The structure of research communities and prior knowledge improves prediction of genetic regulatory experiments

Science could solve some of the world's biggest problems. Why aren't governments using it?

Does big data serve policy? Not without context. An experiment with in silico social science

Theoretical and technological building blocks for an innovation accelerator

Large-scale research infrastructure projects: A conceptual review for science policy and management

Combining support for hypotheses over heterogeneous studies with Bayesian Evidence Synthesis: A simulation study

Evidence-informed climate policy: mobilising strategic research and pooling expertise for rapid evidence generation

We must discuss research environments

Developing and refining the methods for a 'one-stop shop' for research evidence about health systems