Abstract:AbstractMuch of the current debate around the impact of patient/public involvement on research focuses on the lack of empirical data. While a number of systematic literature reviews have reported the various ways in which involvement makes a difference to research and the people involved, this evidence has been criticised as being weak and anecdotal. It is argued that robust evidence is still required. This review reflects on the use of quantitative approaches to evaluating impact. It concludes that the statistical evidence is weakened by not paying sufficient attention to the context in which involvement takes place and the way it is carried out. However, if scientific (systematic, quantitative, empirical) approaches are designed in a way to take these factors into account, they might not generate knowledge that is useful beyond the original context. Such approaches might not therefore enhance our understanding of when, why and how involvement makes a difference. In the context of individual research projects where researchers collaborate with patients/the public, researchers often acquire ‘new’ knowledge about life with a health condition. This new understanding can be described as experiential knowledge—‘knowledge in context’—that researchers gain through direct experience of working with patients/the public. On this basis, researchers’ accounts of their experience potentially provide a source of insight and learning to influence others, in the same way that the patient experience helps to shape research. These accounts could be improved by increasing the detail provided about context and mechanism. One of the most important contextual factors that influence the outcome of involvement is the researchers themselves and the skills, assumptions, values and priorities they start with. At the beginning of any research project, the researchers ‘don’t know what they don’t know’ until they involve patients/the public. This means that the impact of involvement within any particular project is somewhat unpredictable. The answer to the question ‘Is involvement worth doing?’ will always be ‘It depends’. Further exploration of the contextual and mechanistic factors which influence outcomes could give a stronger steer to researchers but may never accurately predict any specific impact.Plain English summaryIn recent years, there has been considerable interest in finding out what difference patient and public involvement makes to research projects. The evidence published so far has been criticised for being weak and anecdotal. Some people argue we need robust evidence of impact from scientific studies of involvement. In this review, I consider examples of where impact has been measured using statistical methods. I conclude that the statistical evidence is weak, if the studies do not consider the context in which involvement takes place and the way that it is done. Studies designed to take this into account give us more confidence that the involvement did make a difference to that particular project. They do not tell us whether the same impact will occur in the same way in other projects and therefore have limited value. Researchers gain an understanding of involvement through their direct experience of working with patients and the public. This is ‘knowledge in context’ or ‘insight’ gained in the same way that patients gain expertise through their direct experience of a health condition. This means that detailed accounts of involvement from researchers already provide valuable learning to others, in the same way that patients’ insights help shape research. However, the impact of involvement will always be somewhat unpredictable, because at the start of any project researchers ‘don’t know what they don’t know’—they do not know precisely what problems they might anticipate, until the patients/public tell them.

Identifying stakeholder preferences for communicating impact from medical research: a mixed methods study

Studying How Patient Engagement Influences Research: A Mixed Methods Study

Applying a framework to assess the impact of cardiovascular outcomes improvement research

Using the UK standards for public involvement to evaluate the public involvement sections of annual reports from NIHR managed research centres

Understanding the Influence and Impact of Stakeholder Engagement in Patient-centered Outcomes Research: a Qualitative Study

The Importance of Documenting the Impacts of Health Communication

Evidencing impact: a case study of UK academic perspectives on evidencing research impact

Assessing the impact of healthcare research: A systematic review of methodological frameworks

Does health intervention research have real world policy and practice impacts: testing a new impact assessment tool

Systematic review and narrative synthesis of the impact of Appreciative Inquiry in healthcare

A proposal for using benefit-risk methods to improve the prominence of adverse event results when reporting trials

Public involvement in research: assessing impact through a realist evaluation

Design, Conduct, and Use of Patient Preference Studies in the Medical Product Life Cycle: A Multi-Method Study

Patient and public involvement in hearing research: opportunities, impact and reflections with case studies from the Manchester Centre for Audiology and Deafness

Evaluating public and patient involvement in interventional research – a newly developed checklist (EPPIIC) with application to the COB-MS feasibility trial

What impact has the Centre of Research Excellence in Digestive Health made in the field of gastrointestinal health in Australia and internationally? Study protocol for impact evaluation using the FAIT framework

Valuing the impact of health and social care programs using social return on investment analysis: how have academics advanced the methodology? A systematic review

Assessing the impact of knowledge communication and dissemination strategies targeted at health policy-makers and managers: an overview of systematic reviews

Importance of key stakeholder communication in the implementation of an in-hospital digital medical translation application for heart surgery patients: a qualitative study

‘Is it worth doing?’ Measuring the impact of patient and public involvement in research

Priorities and expectations of researchers, funders, patients and the public regarding the funding of medical research: results from the PERSPECT qualitative study