Abstract:AbstractMuch of the current debate around the impact of patient/public involvement on research focuses on the lack of empirical data. While a number of systematic literature reviews have reported the various ways in which involvement makes a difference to research and the people involved, this evidence has been criticised as being weak and anecdotal. It is argued that robust evidence is still required. This review reflects on the use of quantitative approaches to evaluating impact. It concludes that the statistical evidence is weakened by not paying sufficient attention to the context in which involvement takes place and the way it is carried out. However, if scientific (systematic, quantitative, empirical) approaches are designed in a way to take these factors into account, they might not generate knowledge that is useful beyond the original context. Such approaches might not therefore enhance our understanding of when, why and how involvement makes a difference. In the context of individual research projects where researchers collaborate with patients/the public, researchers often acquire ‘new’ knowledge about life with a health condition. This new understanding can be described as experiential knowledge—‘knowledge in context’—that researchers gain through direct experience of working with patients/the public. On this basis, researchers’ accounts of their experience potentially provide a source of insight and learning to influence others, in the same way that the patient experience helps to shape research. These accounts could be improved by increasing the detail provided about context and mechanism. One of the most important contextual factors that influence the outcome of involvement is the researchers themselves and the skills, assumptions, values and priorities they start with. At the beginning of any research project, the researchers ‘don’t know what they don’t know’ until they involve patients/the public. This means that the impact of involvement within any particular project is somewhat unpredictable. The answer to the question ‘Is involvement worth doing?’ will always be ‘It depends’. Further exploration of the contextual and mechanistic factors which influence outcomes could give a stronger steer to researchers but may never accurately predict any specific impact.Plain English summaryIn recent years, there has been considerable interest in finding out what difference patient and public involvement makes to research projects. The evidence published so far has been criticised for being weak and anecdotal. Some people argue we need robust evidence of impact from scientific studies of involvement. In this review, I consider examples of where impact has been measured using statistical methods. I conclude that the statistical evidence is weak, if the studies do not consider the context in which involvement takes place and the way that it is done. Studies designed to take this into account give us more confidence that the involvement did make a difference to that particular project. They do not tell us whether the same impact will occur in the same way in other projects and therefore have limited value. Researchers gain an understanding of involvement through their direct experience of working with patients and the public. This is ‘knowledge in context’ or ‘insight’ gained in the same way that patients gain expertise through their direct experience of a health condition. This means that detailed accounts of involvement from researchers already provide valuable learning to others, in the same way that patients’ insights help shape research. However, the impact of involvement will always be somewhat unpredictable, because at the start of any project researchers ‘don’t know what they don’t know’—they do not know precisely what problems they might anticipate, until the patients/public tell them.

The Importance of Documenting the Impacts of Health Communication

The Pursuit of Positive Impacts: Translating Longitudinal Cancer Studies into Successful Health Communication Interventions

From impact metrics and open science to communicating research: Journalists' awareness of academic controversies

Does health intervention research have real world policy and practice impacts: testing a new impact assessment tool

The state of health communication research: A content analysis of articles published in Journal of Health Communication and Health Communication (2010-2019)

The Impact of Health Communication Research on Medical and Health Professional Education and Training

Re-thinking research impact: voice, context and power at the interface of science, policy and practice

Identifying stakeholder preferences for communicating impact from medical research: a mixed methods study

Evidence-based research impact praxis: Integrating scholarship and practice to ensure research benefits society

Assessing the impact of healthcare research: A systematic review of methodological frameworks

Advancing the Impact of Critical Qualitative Research on Policy, Practice, and Science

Identifying the impact of diabetes research

Evidencing impact: a case study of UK academic perspectives on evidencing research impact

Achieving Research Impact Through Co‐creation in Community‐Based Health Services: Literature Review and Case Study

What is the impact of a research publication?

Words Matter: Reflective Science Communication and Tradeoffs in Environmental Health Research

‘Is it worth doing?’ Measuring the impact of patient and public involvement in research

Assessing the impact of knowledge communication and dissemination strategies targeted at health policy-makers and managers: an overview of systematic reviews

Extending the Roots of Memorable Messages: A Comprehensive Review and Forecast of Memorable Message Literature and Theory

Questioning the Impact of the Impact Factor. A Brief Review and Future Directions

Embracing complexity and uncertainty to create impact: exploring the processes and transformative potential of co-produced research through development of a social impact model