How papers with doctored images can affect scientific reviews

Sumeet Kulkarni

DOI: https://doi.org/10.1038/d41586-024-00875-2

IF: 64.8

2024-03-30

Nature

Abstract:Scientists compiling a review scan more than 1,000 papers and find troubling images in nearly 20%. Scientists compiling a review scan more than 1,000 papers and find troubling images in nearly 20%.

multidisciplinary sciences

What problem does this paper attempt to address?

Genuine images in 2024

H Holden Thorp,H. Holden Thorp

DOI: https://doi.org/10.1126/science.adn7530

IF: 56.9

2024-01-05

Science

Abstract:In recent years, the research community has become increasingly concerned with issues involving the manipulation of images in scientific papers. Some of these alterations—involving images from experimental techniques such as microscopy, flow cytometry, and western blots—are inadvertent and may not change the conclusions of papers. But in rare cases, some are done deliberately to mislead readers. Image sleuths who can detect these alterations, like the scientific integrity consultant Elisabeth Bik, have risen to prominence, as has the website PubPeer, where many of the detected flaws are posted. High-profile incidents, such as one involving the laboratory of former Stanford University President Marc Tessier-Lavigne, have eroded public confidence in science and harmed careers of investigators who missed doctored images coming from their own laboratories. To address these problems, in 2024, the Science family of journals is adopting the use of Proofig, an artificial intelligence (AI)–powered image-analysis tool, to detect altered images across all six of the journals.

multidisciplinary sciences
Fraudulent studies are undermining the reliability of systematic reviews – a study of the prevalence of problematic images in preclinical studies of depression

Jenny P. Berrío,Otto Kalliokoski

DOI: https://doi.org/10.1101/2024.02.13.580196

2024-02-15

Abstract:Systematic reviews are considered by many to constitute the highest level of scientific evidence. A caveat is that the methods used in a systematic review – combining information from multiple studies – are predicated on all of the reports being truthful. Currently, we do not know how frequent fraudulent studies are in systematic reviews, or how they affect the resulting evidence base. For a systematic review of preclinical studies of depression, we found that potentially fraudulent studies were not only common but also that they biased the findings of the review. In a sample of 1,035 studies, we found that 19 % of peer-reviewed reports displayed data in the form of problematic images. In a majority of the cases, images had been altered or recycled in a way that makes us suspect foul play. Making things worse, these studies reported larger effect sizes, on average, than did studies where we did not identify problems. Counter to commonly held beliefs, reports with problematic images were not cited less or published in lower-impact journals, nor were their authors isolated to any specific geographic area. The sheer prevalence of problematic studies, and the fact that we could not find a simple pattern for identifying them, undermines the validity of systematic reviews within our research field. We suspect that this is symptomatic of a broader problem that needs immediate addressing.

Neuroscience
Fake papers compromise research syntheses

Holly Else

DOI: https://doi.org/10.1126/science.adu8281

IF: 56.9

2024-11-30

Science

Abstract:"Systematic reviews" that aim to extract broad conclusions from many studies are in peril

multidisciplinary sciences
AI-enabled image fraud in scientific publications

Jinjin Gu,Xinlei Wang,Chenang Li,Junhua Zhao,Weijin Fu,Gaoqi Liang,Jing Qiu

DOI: https://doi.org/10.1016/j.patter.2022.100511

2022-07-08

Abstract:Destroying image integrity in scientific papers may result in serious consequences. Inappropriate duplication and fabrication of images are two common misconducts in this aspect. The rapid development of artificial-intelligence technology has brought to us promising image-generation models that can produce realistic fake images. Here, we show that such advanced generative models threaten the publishing system in academia as they may be used to generate fake scientific images that cannot be effectively identified. We demonstrate the disturbing potential of these generative models in synthesizing fake images, plagiarizing existing images, and deliberately modifying images. It is very difficult to identify images generated by these models by visual inspection, image-forensic tools, and detection tools due to the unique paradigm of the generative models for processing images. This perspective reveals vast risks and arouses the vigilance of the scientific community on fake scientific images generated by artificial intelligence (AI) models.
[Scientific misconduct: A major threat for medical research]

F Barde,N Peiffer-Smadja,A de La Blanchardière

DOI: https://doi.org/10.1016/j.revmed.2020.02.004

Abstract:Scientific misconduct (fabrication, falsification, and plagiarism) and detrimental research practices (selective reporting of data, inappropriate citation practice, ghostwriting) are admitted respectively by 2 % and 33 % of researchers. The consequences of scientific misconduct and detrimental research practices are disastrous, both for the doctors, who are the most affected researchers in view of the number of retracted articles, and for the patients, victims of false information that may have health consequences. In order to fight against the causes (promotion of doctors and allocation of resources to clinical wards and laboratories on purely quantitative research criteria, lack of training in scientific integrity in medical studies, heterogenous quality of reviewing, legal impunity), there are legislative, academic, technological and editorial solutions, but radical and urgent cultural change is needed first.
Widespread misidentification of SEM instruments in the peer-reviewed materials science and engineering literature

Reese AK Richardson,Jeonghyun Moon,Spencer S Hong,Luís A Nunes Amaral

2024-08-27

Abstract:Materials science and engineering (MSE) research has, for the most part, escaped the doubts raised about the reliability of the scientific literature by recent large-scale replication studies in psychology and cancer biology. However, users on post-publication peer review sites have recently identified dozens of articles where the make and model of the scanning electron microscope (SEM) listed in the text of the paper does not match the instrument's metadata visible in the images in the published article. In order to systematically investigate this potential risk to the MSE literature, we develop a semi-automated approach to scan published figures for this metadata and check it against the SEM instrument identified in the text. Starting from an exhaustive set of 1,067,102 articles published since 2010 in 50 journals with impact factors ranging from 2 to 24, we identify 11,314 articles for which SEM make and model can be identified in an image's metadata. For 21.2% of those articles, the image metadata does not match the SEM manufacturer or model listed in the text and, for another 24.7%, at least some of the instruments used in the study are not reported. Unexplained patterns common to many of these articles suggest the involvement of paper mills, organizations that mass-produce, sell authorship on, and publish fraudulent scientific manuscripts at scale.

Physics and Society,Digital Libraries
Scientific Image Tampering Detection Based On Noise Inconsistencies: A Method And Datasets

Ziyue Xiang,Daniel Ernesto Acuna

DOI: https://doi.org/10.21203/rs.2.22943/v1

2020-02-07

Abstract:Abstract Background: Scientific image tampering is a problem that affects not only authors but also the general perception of the research community. Although previous researchers have developed methods to identify tampering in natural images, these methods may not thrive under the scientific setting as scientific images have different statistics, format, quality, and intentions. Methods: We propose a scientific-image specific tampering detection method based on noise inconsistencies, which is capable of learning and generalizing to different fields of science. We train and test our method on a new dataset of manipulated western blot and microscopy imagery, which aims at emulating problematic images in science. Results: With an average AUC score of 0.927 and an average F1 score of 0.770, it is shown that our method can detect various types of image manipulation in different scenarios robustly. It outperforms other existing general-purpose image tampering detection schemes. Conclusions: The experiment results shows that our method is capable of detecting manipulations in scientific images in a more reliable manner. We discuss applications beyond these two types of images and suggest next steps for making detection of problematic images a systematic step in peer review and science in general. Keywords: Scientific images; Digital image forensics; Noise inconsistency; Scientific image manipulation dataset
Retracted: Using Color Difference Compensation Method to Balance and Repair the Image of Art Design

Complexity

DOI: https://doi.org/10.1155/2024/9803053

IF: 2.3

2024-01-26

Complexity

Abstract:This article has been retracted by Hindawi following an investigation undertaken by the publisher [1]. This investigation has uncovered evidence of one or more of the following indicators of systematic manipulation of the publication process: (1) Discrepancies in scope (2) Discrepancies in the description of the research reported (3) Discrepancies between the availability of data and the research described (4) Inappropriate citations (5) Incoherent, meaningless and/or irrelevant content included in the article (6) Manipulated or compromised peer review

mathematics, interdisciplinary applications,multidisciplinary sciences
Learning to identify image manipulations in scientific publications

Ghazal Mazaheri,Kevin Urrutia Avila,Amit K. Roy-Chowdhury

DOI: https://doi.org/10.48550/arXiv.2102.01874

2021-02-03

Abstract:Adherence to scientific community standards ensures objectivity, clarity, reproducibility, and helps prevent bias, fabrication, falsification, and plagiarism. To help scientific integrity officers and journal/publisher reviewers monitor if researchers stick with these standards, it is important to have a solid procedure to detect duplication as one of the most frequent types of manipulation in scientific papers. Images in scientific papers are used to support the experimental description and the discussion of the findings. Therefore, in this work we focus on detecting the duplications in images as one of the most important parts of a scientific paper. We propose a framework that combines image processing and deep learning methods to classify images in the articles as duplicated or unduplicated ones. We show that our method leads to a 90% accuracy rate of detecting duplicated images, a ~ 13% improvement in detection accuracy in comparison to other manipulation detection methods. We also show how effective the pre-processing steps are by comparing our method to other state-of-art manipulation detectors which lack these steps.

Computer Vision and Pattern Recognition
High retraction rate of Chinese articles: it is time to do something about academic misconduct

Yu Xiao,Jia Chen,Xiao-hong Wu,Qin-ming Qiu

DOI: https://doi.org/10.1136/postgradmedj-2021-140853

2021-09-06

Postgraduate Medical Journal

Abstract:As researchers, we found the article, 'Peer reviews. A peer reviewer's view'1 thought provoking. The author states that a detailed analysis of retracted papers reveals that faked peer reviews are not uncommon. Shockingly, 75% of 250 retracted papers that had faked peer review were written by Chinese authors.2 Nowadays, because researchers are eager to be recognised and cited, the peer review system is greatly challenged. Between 2007 and 2018, the retraction rate of Chinese authors' Science Citation Index (SCI) papers was the highest in the world, reaching 22.7/10 000 papers, five times that of the USA.3 Retractions were issued because of fraud, inconsistent reporting, plagiarism, mistakes, duplication, legal and ethical concerns, and disputed authorship. A study shows that the article types which are frequently retracted are original research works, randomised trials and reviews, and there is a strong association between the retracted items and...

medicine, general & internal
Questionable papers in citation databases as an issue for literature review

Mehdi Dadkhah,Mohammad Lagzian,Glenn Borchardt

DOI: https://doi.org/10.1007/s12079-016-0370-6

2017-02-18

Journal of Cell Communication and Signaling

Abstract:In recent years, the academic world has been faced with much academic misconduct. Examples involve plagiarizing papers, manipulating data, and launching predatory or hijacked journals. The literature exposing these activities is growing exponentially, and so is the presentation of criteria or guidelines for counteracting the problem. Most of the research is focused on predatory or hijacked journal detection and providing suitable warnings. Overlooked in all this is the fact that papers published in these journals are questionable, but nevertheless show up in standard citation databases. We need some way to flag them so future researchers will be aware of their questionable nature and prevent their use in literature review.

cell biology
Dana-Farber retractions: meet the blogger who spotted problems in dozens of cancer papers

Max Kozlov

DOI: https://doi.org/10.1038/d41586-024-00202-9

IF: 64.8

2024-01-26

Nature

Abstract:Nature talks to Sholto David about his process for flagging image manipulation and his tips for scientists under scrutiny. Nature talks to Sholto David about his process for flagging image manipulation and his tips for scientists under scrutiny.

multidisciplinary sciences
SILA: a system for scientific image analysis

Daniel Moreira,João Phillipe Cardenuto,Ruiting Shao,Sriram Baireddy,Davide Cozzolino,Diego Gragnaniello,Wael Abd-Almageed,Paolo Bestagini,Stefano Tubaro,Anderson Rocha,Walter Scheirer,Luisa Verdoliva,Edward Delp

DOI: https://doi.org/10.1038/s41598-022-21535-3

IF: 4.6

2022-11-01

Scientific Reports

Abstract:A great deal of the images found in scientific publications are retouched, reused, or composed to enhance the quality of the presentation. In most instances, these edits are benign and help the reader better understand the material in a paper. However, some edits are instances of scientific misconduct and undermine the integrity of the presented research. Determining the legitimacy of edits made to scientific images is an open problem that no current technology can perform satisfactorily in a fully automated fashion. It thus remains up to human experts to inspect images as part of the peer-review process. Nonetheless, image analysis technologies promise to become helpful to experts to perform such an essential yet arduous task. Therefore, we introduce SILA, a system that makes image analysis tools available to reviewers and editors in a principled way. Further, SILA is the first human-in-the-loop end-to-end system that starts by processing article PDF files, performs image manipulation detection on the automatically extracted figures, and ends with image provenance graphs expressing the relationships between the images in question, to explain potential problems. To assess its efficacy, we introduce a dataset of scientific papers from around the globe containing annotated image manipulations and inadvertent reuse, which can serve as a benchmark for the problem at hand. Qualitative and quantitative results of the system are described using this dataset.

multidisciplinary sciences
Scientific integrity and fraud in radiology research

Robert M Kwee,Maan T Almaghrabi,Thomas C Kwee

DOI: https://doi.org/10.1016/j.ejrad.2022.110553

Abstract:Purpose: To investigate the view of radiologists on the integrity of their own and their colleagues' scientific work. Materials and methods: Corresponding authors of articles that were published in 12 general radiology journals in 2021 were invited to participate in a survey on scientific integrity. Results: A total of 219 (6.2 %) of 3,511 invited corresponding authors participated. Thirteen (5.9 %) respondents reported having committed scientific fraud, and 60 (27.4 %) witnessed or suspect scientific fraud among their departmental members in the past 5 years. Misleading reporting (32.2 %), duplicate/redundant publication (26.3 %), plagiarism (15.3 %), and data manipulation/falsification (13.6 %) were the most commonly reported types of scientific fraud. Publication bias exists according to 184 (84.5 %) respondents, and 89 (40.6 %) respondents had honorary authors on their publications in the past 5 years. General confidence in the integrity of scientific publications ranged between 2 and 10 (median: 8) on a 0-10 point scale. Common topics of interest and concern among respondents were authorship criteria and assignments, perverse incentives (including the influence of money, funding, and academic promotions on the practice of research), and poorly performed research without intentional fraud. Conclusion: Radiology researchers reported that scientific fraud and other undesirable practices such as publication bias and honorary authorship are relatively common. Their general confidence in the scientific integrity of published work was relatively high, but far from perfect. These data may trigger stakeholders in the radiology community to place scientific integrity higher on the agenda, and to initiate cultural and policy reforms to remove perverse research incentives.
Finding the truth in science

Elisabeth M. Bik

DOI: https://doi.org/10.1038/s41568-024-00702-w

IF: 78.5

2024-05-17

Nature Reviews Cancer

Abstract:In a 'publish or perish' culture, some scientists may resort to questionable research practices or even fraud. Scientific paper mills and artificial intelligence increasingly threaten the pursuit of truth in science. Structural changes, including heightened scrutiny of papers and authorship and better funding, are needed to ensure scientific integrity.

oncology
Retracted: Processing Methods for Digital Image Data Based on the Geographic Information System

Complexity

DOI: https://doi.org/10.1155/2024/9815913

IF: 2.3

2024-01-26

Complexity

Abstract:This article has been retracted by Hindawi following an investigation undertaken by the publisher [1]. This investigation has uncovered evidence of one or more of the following indicators of systematic manipulation of the publication process: (1) Discrepancies in scope (2) Discrepancies in the description of the research reported (3) Discrepancies between the availability of data and the research described (4) Inappropriate citations (5) Incoherent, meaningless and/or irrelevant content included in the article (6) Manipulated or compromised peer review

mathematics, interdisciplinary applications,multidisciplinary sciences
Unveiling scientific articles from paper mills with provenance analysis

João Phillipe Cardenuto,Daniel Moreira,Anderson Rocha

DOI: https://doi.org/10.1371/journal.pone.0312666

IF: 3.7

2024-10-31

PLoS ONE

Abstract:The increasing prevalence of fake publications created by paper mills poses a significant challenge to maintaining scientific integrity. While integrity analysts typically rely on textual and visual clues to identify fake articles, determining which papers merit further investigation can be akin to searching for a needle in a haystack, as these fake publications have non-related authors and are published on non-related venues. To address this challenge, we developed a new methodology for provenance analysis, which automatically tracks and groups suspicious figures and documents. Our approach groups manuscripts from the same paper mill by analyzing their figures and identifying duplicated and manipulated regions. These regions are linked and organized in a provenance graph, providing evidence of systematic production. We tested our solution on a paper mill dataset of hundreds of documents and also on a larger version of the dataset that deliberately included thousands of documents intentionally selected to distract our method. Our approach successfully identified and linked systematically produced articles on both datasets by pinpointing the figures they reused and manipulated from one another. The technique herein proposed offers a promising solution to identify fraudulent manuscripts, and it could be a valuable tool for supporting scientific integrity.

multidisciplinary sciences
Decontamination of the scientific literature

Guillaume Cabanac

DOI: https://doi.org/10.48550/arXiv.2210.15912

2022-10-28

Digital Libraries

Abstract:Research misconduct and frauds pollute the scientific literature. Honest errors and malevolent data fabrication, image manipulation, journal hijacking, and plagiarism passed peer review unnoticed. Problematic papers deceive readers, authors citing them, and AI-powered literature-based discovery. Flagship publishers accepted hundreds flawed papers despite claiming to enforce peer review. This application ambitions to decontaminate the scientific literature using curative and preventive actions.
Artificial Intelligence for COVID-19 Detection in Medical Imaging—Diagnostic Measures and Wasting—A Systematic Umbrella Review

Paweł Jemioło,Dawid Storman,Patryk Orzechowski

DOI: https://doi.org/10.3390/jcm11072054

IF: 3.9

2022-04-06

Journal of Clinical Medicine

Abstract:The COVID-19 pandemic has sparked a barrage of primary research and reviews. We investigated the publishing process, time and resource wasting, and assessed the methodological quality of the reviews on artificial intelligence techniques to diagnose COVID-19 in medical images. We searched nine databases from inception until 1 September 2020. Two independent reviewers did all steps of identification, extraction, and methodological credibility assessment of records. Out of 725 records, 22 reviews analysing 165 primary studies met the inclusion criteria. This review covers 174,277 participants in total, including 19,170 diagnosed with COVID-19. The methodological credibility of all eligible studies was rated as critically low: 95% of papers had significant flaws in reporting quality. On average, 7.24 (range: 0–45) new papers were included in each subsequent review, and 14% of studies did not include any new paper into consideration. Almost three-quarters of the studies included less than 10% of available studies. More than half of the reviews did not comment on the previously published reviews at all. Much wasting time and resources could be avoided if referring to previous reviews and following methodological guidelines. Such information chaos is alarming. It is high time to draw conclusions from what we experienced and prepare for future pandemics.

medicine, general & internal
Prevalence of nonsensical algorithmically generated papers in the scientific literature

Guillaume Cabanac,Cyril Labbé

DOI: https://doi.org/10.1002/asi.24495

2021-05-25

Journal of the Association for Information Science and Technology

Abstract:<p>In 2014 leading publishers withdrew more than 120 nonsensical publications automatically generated with the SCIgen program. Casual observations suggested that similar problematic papers are still published and sold, without follow-up retractions. No systematic screening has been performed and the prevalence of such nonsensical publications in the scientific literature is unknown. Our contribution is 2-fold. First, we designed a detector that combs the scientific literature for grammar-based computer-generated papers. Applied to SCIgen, it has a 83.6% precision. Second, we performed a scientometric study of the 243 detected SCIgen-papers from 19 publishers. We estimate the prevalence of SCIgen-papers to be 75 per million papers in Information and Computing Sciences. Only 19% of the 243 problematic papers were dealt with: formal retraction (12) or silent removal (34). Publishers still serve and sometimes sell the remaining 197 papers without any caveat. We found evidence of citation manipulation via edited SCIgen bibliographies. This work reveals metric gaming up to the point of absurdity: fraudsters publish nonsensical algorithmically generated papers featuring genuine references. It stresses the need to screen papers for nonsense before peer-review and chase citation manipulation in published papers. Overall, this is yet another illustration of the harmful effects of the pressure to publish or perish.</p>

information science & library science,computer science, information systems

How papers with doctored images can affect scientific reviews

Genuine images in 2024

Fraudulent studies are undermining the reliability of systematic reviews – a study of the prevalence of problematic images in preclinical studies of depression

Fake papers compromise research syntheses

AI-enabled image fraud in scientific publications

[Scientific misconduct: A major threat for medical research]

Widespread misidentification of SEM instruments in the peer-reviewed materials science and engineering literature

Scientific Image Tampering Detection Based On Noise Inconsistencies: A Method And Datasets

Retracted: Using Color Difference Compensation Method to Balance and Repair the Image of Art Design

Learning to identify image manipulations in scientific publications

High retraction rate of Chinese articles: it is time to do something about academic misconduct

Questionable papers in citation databases as an issue for literature review

Dana-Farber retractions: meet the blogger who spotted problems in dozens of cancer papers

SILA: a system for scientific image analysis

Scientific integrity and fraud in radiology research

Finding the truth in science

Retracted: Processing Methods for Digital Image Data Based on the Geographic Information System

Unveiling scientific articles from paper mills with provenance analysis

Decontamination of the scientific literature

Artificial Intelligence for COVID-19 Detection in Medical Imaging—Diagnostic Measures and Wasting—A Systematic Umbrella Review

Prevalence of nonsensical algorithmically generated papers in the scientific literature