Abstract:Background Bias in the publication and reporting of research findings (referred to as publication and related bias here) poses a major threat in evidence synthesis and evidence-based decision-making. Although this bias has been well documented in clinical research, little is known about its occurrence and magnitude in health services and delivery research. Objectives To obtain empirical evidence on publication and related bias in quantitative health services and delivery research; to examine current practice in detecting/mitigating this bias in health services and delivery research systematic reviews; and to explore stakeholders’ perception and experiences concerning such bias. Methods The project included five distinct but interrelated work packages. Work package 1 was a systematic review of empirical and methodological studies. Work package 2 involved a survey (meta-epidemiological study) of randomly selected systematic reviews of health services and delivery research topics ( n = 200) to evaluate current practice in the assessment of publication and outcome reporting bias during evidence synthesis. Work package 3 included four case studies to explore the applicability of statistical methods for detecting such bias in health services and delivery research. In work package 4 we followed up four cohorts of health services and delivery research studies (total n = 300) to ascertain their publication status, and examined whether publication status was associated with statistical significance or perceived ‘positivity’ of study findings. Work package 5 involved key informant interviews with diverse health services and delivery research stakeholders ( n = 24), and a focus group discussion with patient and service user representatives ( n = 8). Results We identified only four studies that set out to investigate publication and related bias in health services and delivery research in work package 1. Three of these studies focused on health informatics research and one concerned health economics. All four studies reported evidence of the existence of this bias, but had methodological weaknesses. We also identified three health services and delivery research systematic reviews in which findings were compared between published and grey/unpublished literature. These reviews found that the quality and volume of evidence and effect estimates sometimes differed significantly between published and unpublished literature. Work package 2 showed low prevalence of considering/assessing publication (43%) and outcome reporting (17%) bias in health services and delivery research systematic reviews. The prevalence was lower among reviews of associations than among reviews of interventions. The case studies in work package 3 highlighted limitations in current methods for detecting these biases due to heterogeneity and potential confounders. Follow-up of health services and delivery research cohorts in work package 4 showed positive association between publication status and having statistically significant or positive findings. Diverse views concerning publication and related bias and insights into how features of health services and delivery research might influence its occurrence were uncovered through the interviews with health services and delivery research stakeholders and focus group discussion conducted in work package 5. Conclusions This study provided prima facie evidence on publication and related bias in quantitative health services and delivery research. This bias does appear to exist, but its prevalence and impact may vary depending on study characteristics, such as study design, and motivation for conducting the evaluation. Emphasis on methodological novelty and focus beyond summative assessments may mitigate/lessen the risk of such bias in health services and delivery research. Methodological and epistemological diversity in health services and delivery research and changing landscape in research publication need to be considered when interpreting the evidence. Collection of further empirical evidence and exploration of optimal health services and delivery research practice are required. Study registration This study is registered as PROSPERO CRD42016052333 and CRD42016052366. Funding This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research ; Vol. 8, No. 33. See the NIHR Journals Library website for further project information.

Ein Bias kommt selten allein: Wie (auch) Forschung Panik schürt

Publication Bias and Dissemination of Clinical Research

Kommentar zu "JUNGES FORUM – Kognitive Verzerrung in der Radiologie"

Common method biases in behavioral research: A critical review of the literature and recommended remedies.

Who is at risk of bias? Examining dispositional differences in motivated science reception

Biases in Scholarly Recommender Systems: Impact, Prevalence, and Mitigation

Potential types of bias when estimating causal effects in environmental research and how to interpret them

Das Konfidenz- / Signifikanzniveau impliziert ein bestimmtes Kostenverhältnis zwischen Fehler 1. Art und Fehler 2. Art

Publication and related bias in quantitative health services and delivery research: a multimethod study

Do You Think It's Biased? How To Ask For The Perception Of Media Bias

The hidden side of animal cognition research: Scientists' attitudes toward bias, replicability and scientific practice

Examining bias perpetuation in academic search engines: an algorithm audit of Google and Semantic Scholar

Conceptual and Methodological Pitfalls in Experimental Studies: An Overview, and the Case of Alzheimer’s Disease

Footprint of publication selection bias on meta‐analyses in medicine, environmental sciences, psychology, and economics

The Meaningfulness of Effect Sizes in Psychological Research: Differences Between Sub-Disciplines and the Impact of Potential Biases

Unveiling the Hidden Agenda: Biases in News Reporting and Consumption

Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis

Cognitive Biases in Fact-Checking and Their Countermeasures: A Review

JUNGES FORUM – Kognitive Verzerrung in der Radiologie

How to be fair? A study of label and selection bias

Overconfidence and Prejudice