Peer review demystified: part 1

DOI: https://doi.org/10.1038/s41592-024-02225-6

IF: 48

2024-03-13

Nature Methods

Abstract:Peer review is at the heart of publishing scientific papers. In this first installment of a two-part Editorial, we explain how we manage the process at Nature Methods.

biochemical research methods

What problem does this paper attempt to address?

Peer review demystified: part 2

DOI: https://doi.org/10.1038/s41592-024-02258-x

IF: 48

2024-04-13

Nature Methods

Abstract:Nature Methods, Published online: 12 April 2024; doi:10.1038/s41592-024-02258-x We continue our explanation of the peer review process at Nature Methods.

biochemical research methods
The Peer Review Process: Past, Present, and Future

John A. Drozdz,Michael R. Ladomery

DOI: https://doi.org/10.3389/bjbs.2024.12054

2024-06-17

British Journal of Biomedical Science

Abstract:The peer review process is a fundamental aspect of modern scientific paper publishing, underpinning essential quality control. First conceptualised in the 1700s, it is an iterative process that aims to elevate scientific literature to the highest standards whilst preventing publication of scientifically unsound, potentially misleading, and even plagiarised information. It is widely accepted that the peer review of scientific papers is an irreplaceable and fundamental aspect of the research process. However, the rapid growth of research and technology has led to a huge increase in the number of publications. This has led to increased pressure on the peer review system. There are several established peer review methodologies, ranging from single and double blind to open and transparent review, but their implementation across journals and research fields varies greatly. Some journals are testing entirely novel approaches (such as collaborative reviews), whilst others are piloting changes to established methods. Given the unprecedented growth in publication numbers, and the ensuing burden on journals, editors, and reviewers, it is imperative to improve the quality and efficiency of the peer review process. Herein we evaluate the peer review process, from its historical origins to current practice and future directions.

medical laboratory technology
Scientific peer review in the modern era: A comprehensive guide

NimratKaur Sandhu,Rhea Wason,Ashish Goel

DOI: https://doi.org/10.4103/0973-3698.364681

2022-12-22

Indian Journal of Rheumatology

Abstract:The scientific peer review process involves scrutinizing the work of researchers by a panel of scientific experts who are selected by the journal to which they have submitted a manuscript. It ensures that only the best ideas and the highest quality of research are circulated through the publications in the scientific journals. The concept of peer review preceded the development of the scientific journal, has co-evolved with the scientific method, and has now become irreplaceably integrated with it. The traditional method of peer review involved multiple submissions and revisions and was criticized for being time-consuming and biased with the reports of professional misconduct even among the renowned journals. Several guidelines now ensure a more ethical, consistent and rigorous review process. Novel methods of review such as the use of registered reports and the dynamic review methods are also being explored by publishers to deal with these issues. Although the role of peer review as a gatekeeper for high-quality research has been well established, its value in future research is rapidly evolving. Newer methods and adherence to guidelines will go a long way in improving this imperfect pillar of science.

English Else
An overview of the peer review process in biomedical sciences

Edward Miller,Michael James Weightman,Ashna Basu,Andrew Amos,Vlasios Brakoulias

DOI: https://doi.org/10.1177/10398562241231460

2024-02-10

Australasian Psychiatry

Abstract:Australasian Psychiatry, Ahead of Print. ObjectiveThis paper aims to provide an introductory resource for beginner peer reviewers in psychiatry and the broader biomedical science field. It will provide a concise overview of the peer review process, alongside some reviewing tips and tricks.ConclusionThe peer review process is a fundamental aspect of biomedical science publishing. The model of peer review offered varies between journals and usually relies on a pool of volunteers with differing levels of expertise and scope. The aim of peer review is to collaboratively leverage reviewers' collective knowledge with the objective of increasing the quality and merit of published works. The limitations, methodology and need for transparency in the peer review process are often poorly understood. Although imperfect, the peer review process provides some degree of scientific rigour by emphasising the need for an ethical, comprehensive and systematic approach to reviewing articles. Contributions from junior reviewers can add significant value to manuscripts.

psychiatry
The Peer Review Process

L Denise Willis

DOI: https://doi.org/10.4187/respcare.11838

2024-03-28

Respiratory Care

Abstract:The purpose of peer review is to evaluate the scientific merit of the submitted work and to assess suitability for publication. This process is intended to provide an unbiased, independent critique to ensure publication of high-quality manuscripts that demonstrate validity and reliability. Reviewers are subject-matter experts who volunteer their time to participate in peer review. A proper review provides constructive and helpful feedback in a timely manner that authors can use to improve both current and future work. When given the opportunity to revise, authors should carefully consider all comments and adequately address all concerns. This paper provides guidance to clinicians for both aspects of the peer review process: participating as a reviewer and responding to reviewer feedback.

respiratory system,critical care medicine
Motivating and Maintaining Ethics, Equity, Effectiveness, Efficiency, and Expertise in Peer Review

Adam Craig,Christina Lee,Nithyaa Bala,Carl Taswell,

DOI: https://doi.org/10.48085/i5b147d9d

2022-06-30

Brainiacs Journal of Brain Imaging And Computing Sciences

Abstract:Scientists who engage in science and the scientific endeavor should seek truth with conviction of morals and commitment to ethics. While the number of publications continues to increase, the number of retractions has increased at a faster rate. Journals publish fraudulent research papers despite claims of peer review and adherence to publishing ethics. Nevertheless, appropriate ethical peer review will remain a gatekeeper when selecting research manuscripts in scholarly publishing and approving research applications for grant funding. However, this peer review must become more open, fair, transparent, equitable, and just with new recommendations and guidelines for reproducible and accountable reviews that support and promote fair citation and citational justice. We should engineer this new peer-review process with modern informatics technology and information science to provide and defend better safeguards for truth and integrity, to clarify and maintain the provenance of information and ideas, and to rebuild and restore trust in scholarly research institutions. Indeed, this new approach will be necessary in the current post-truth era to counter the ease and speed with which mis-information, dis-information, anti-information, caco-information, and mal-information spread through the internet, web, news, and social media. The most important question for application of new peer-review methods to these information wars should be ‘Who does what when?’ in support of reproducible and accountable reviews. Who refers to the authors, reviewers, editors, and publishers as participants in the review process. What refers to disclosure of the participants' identities, the material content of author manuscripts and reviewer commentaries, and other communications between authors and reviewers. When refers to tracking the sequential points in time for which disclosure of whose identity, which content, and which communication at which step of the peer-review process for which audience of readers and reviewers. We believe that quality peer review, and peer review of peer review, must be motivated and maintained by elevating their status and prestige to an art and a science. Both peer review itself and peer review analyses of peer reviews should be incentivised by publishing peer reviews as citable references separately from the research report reviewed while crossreferenced and crosslinked to the report reviewed.
How to be a good reviewer: A step-by-step guide for approaching peer review of a scientific manuscript

Ahmad R Sedaghat,Manuel Bernal-Sprekelsen,Wytske J Fokkens,Timothy L Smith,Michael G Stewart,Romaine F Johnson

DOI: https://doi.org/10.1002/lio2.1266

2024-06-04

Abstract:Objectives: The peer review process is critical to maintaining quality, reliability, novelty, and innovation in the scientific literature. However, the teaching of scientific peer review is rarely a component of formal scientific or clinical training, and even the most experienced peer reviewers express interest in continuing education. The objective of this review article is to summarize the collective perspectives of experienced journal editors about how to be a good reviewer in a step-by-step guide that can serve as a resource for the performance of peer review of a scientific manuscript. Methods: This is a narrative review. Results: A review of the history and an overview of the modern-day peer review process are provided with attention to the role played by the reviewer, including important reasons for involvement in scientific peer review. The general components of a scientific peer review are described, and a model for how to structure a peer review report is provided. These concepts are also summarized in a reviewer checklist that can be used in real-time to develop and double-check one's reviewer report before submitting it. Conclusions: Peer review is a critically important service for maintaining quality in the scientific literature. Peer review of a scientific manuscript and the associated reviewer's report should assess specific details related to the accuracy, validity, novelty, and interpretation of a study's results. We hope that this article will serve as a resource and guide for reviewers of all levels of experience in the performance of peer review of a scientific manuscript.
Peer Review and Medical Journals

Kenneth Nugent,Christopher J Peterson

DOI: https://doi.org/10.1177/21501319241252235

Abstract:Journal editors depend on peer reviewers to make decisions about submitted manuscripts. These reviewers help evaluate the methods, the results, the discussion of the results, and the overall organization and presentation of the manuscript. In addition, reviewers can help identify important mistakes and possible misconduct. Editors frequently have difficulty obtaining enough peer reviews which are submitted in a timely manner. This increases the workload of editors and journal managers and potentially delays the publication of clinical and research studies. This commentary discusses of the importance of peer reviews and make suggestions which potentially can increase the participation of academic faculty and researchers in this important activity.
Mastering the scientific peer review process: tips for young authors from a young senior editor

Evgenios Agathokleous

DOI: https://doi.org/10.1007/s11676-021-01388-8

2021-09-16

Journal of Forestry Research

Abstract:Are you a student at a higher institution or an early-career researcher who is striving to understand and master the peer review process so to increase the odds of getting a paper published in the Journal of Forestry Research or another reputable, peer-reviewed, scientific journal? In this paper, a young, senior editor provides a handbook of the peer review process based on his decadal experience in scientific publishing. He covers major information you need to know during the entire process, from selecting journals to completing the proofing of your accepted paper. He introduces key points for consideration, such as avoidance of predatory journals, dubious research practices and ethics, interaction with peers, reviewers, and editors, and the pursuit of aretê. Finally, he points out some common statistical errors and misconceptions, such as P hacking and incorrect effect size inference. He hopes that this paper will enhance your understanding and knowledge of the peer-review process.

forestry
The Importance and Responsibilities of Reviewers

_ _

DOI: https://doi.org/10.1175/jcli-d-24-0451.1

IF: 5.38

2024-09-19

Journal of Climate

Abstract:Scholarly publishing depends on careful evaluation of manuscripts. While editors can check the basics of a submission, they do not have the expertise to assess every manuscript that crosses their desks. Peer reviewers are subject-matter experts who volunteer their time both to advise editors on the suitability of a manuscript for publication and to provide guidance to authors in improving the accuracy and readability of their study (Schultz 2022). For these services, we as editors are truly grateful: we could not publish high-quality journals without you.

meteorology & atmospheric sciences
How I am working as a reviewer?

Andrzej Grzybowski

DOI: https://doi.org/10.1111/aos.16277

2024-01-01

Acta Ophthalmologica

Abstract:Peer review is designed to evaluate the validity, quality and often the originality of articles for publication. Its aim is to provide the integrity of science by filtering out invalid or poor‐quality submissions. Moreover, the process of peer review adds value to the article by helping to improve that work. 90% of researchers feel that peer review improves the quality of their published paper. Peer reviewing is a critical, and constructive evaluation of submission, particularly including analysis of the paper structure and presentation, methodology strength and clarity; and originality of the study. There are some benefits related with reviewing. It makes a person better writer and better author of future scientific pieces. It helps to think critically about own research and become a better researcher. It is often a sign and proof of the reputation within the field. High‐quality reviews are noticed and appreciated by the editors, who often will invite reviewer to be more involved in editorial work, as an author of editorials or editorial board member. Very often journals invite new editorial members only with some previous history of reviewing for the journal. Reviewers also have often access to emerging ideas and trends much earlier than others.

ophthalmology
A history and development of peer-review process

Siladitya Jana

2019-07-12

Abstract:The paper shows the importance of peer review process in the scholarly communication system and discusses both the closed and the newly emerging open peer review models. It also examines the peer review system at the scholarly academies or societies in their nomination systems for prizes, rewards, etc. It also discusses the various facets of the newly developed open peer review models now prevalent in various journals. The paper may help to understand and appreciate the role played by peer review in the scholarly communication system and the efforts being made to make it more transparent.

History,Education,Political Science
Avoiding a Tragedy of the Commons in the Peer Review Process

D Sculley,Jasper Snoek,Alex Wiltschko

DOI: https://doi.org/10.48550/arXiv.1901.06246

2018-12-18

Abstract:Peer review is the foundation of scientific publication, and the task of reviewing has long been seen as a cornerstone of professional service. However, the massive growth in the field of machine learning has put this community benefit under stress, threatening both the sustainability of an effective review process and the overall progress of the field. In this position paper, we argue that a tragedy of the commons outcome may be avoided by emphasizing the professional aspects of this service. In particular, we propose a rubric to hold reviewers to an objective standard for review quality. In turn, we also propose that reviewers be given appropriate incentive. As one possible such incentive, we explore the idea of financial compensation on a per-review basis. We suggest reasonable funding models and thoughts on long term effects.

Computers and Society,Digital Libraries,Machine Learning
The delicate nature of a constructive peer review: pearls from the editorial board

Lee, Keng Siang

DOI: https://doi.org/10.1007/s10143-024-03047-y

IF: 2.8

2024-10-24

Neurosurgical Review

Abstract:Peer review stands as a cornerstone of academic publishing, especially in the era of evidence-based neurosurgery – the scientific literature relies on proficient peer reviewers. Providing a constructive peer review is an art and learned skill that requires knowledge of study design and expertise in the neurosurgical subspeciality. Peer reviewers guard against arbitrary decision-making and are essential in ensuring that published manuscripts are of the highest quality. However, there remains a scarcity in the formal training relating to the peer review process. The objective of this article is therefore to shed light on this process through the lens of the Editorial Board. We encourage our invited peer reviewers to make use of this guide when appraising potential manuscripts.

surgery,clinical neurology
Knowledge behind the scenes of a peer-reviewed journal helps in publishing your manuscript

Francesca Cappitelli,Federica Villa

DOI: https://doi.org/10.1186/s13213-024-01777-4

2024-10-18

Annals of Microbiology

Abstract:This paper provides insights into the paper submission and review processes and the possible reasons behind paper acceptance or rejection. The main target of this paper is emerging researchers who are writing their first scientific papers in peer-reviewed journals. This manuscript gives clues on how to select a journal and what a preprint is, introduces authorship and the roles of editors, reviewers, and publishers, sheds light on the editorial process, and briefly introduces some ethical issues arising in scholarly publishing. The content is based on a course offered annually to doctoral students at the University of Milan and keeps into account frequently asked questions. The authors of this paper are an editor-in-chief and an associated editor of scientific journals, respectively.

microbiology,biotechnology & applied microbiology
Understanding Peer Review of Software Engineering Papers

Neil A. Ernst,Jeffrey C. Carver,Daniel Mendez,Marco Torchiano

DOI: https://doi.org/10.48550/arXiv.2009.01209

2021-06-18

Abstract:Peer review is a key activity intended to preserve the quality and integrity of scientific publications. However, in practice it is far from perfect. We aim at understanding how reviewers, including those who have won awards for reviewing, perform their reviews of software engineering papers to identify both what makes a good reviewing approach and what makes a good paper. We first conducted a series of in-person interviews with well-respected reviewers in the software engineering field. Then, we used the results of those interviews to develop a questionnaire used in an online survey and sent out to reviewers from well-respected venues covering a number of software engineering disciplines, some of whom had won awards for their reviewing efforts. We analyzed the responses from the interviews and from 175 reviewers who completed the online survey (including both reviewers who had won awards and those who had not). We report on several descriptive results, including: 45% of award-winners are reviewing 20+ conference papers a year, while 28% of non-award winners conduct that many. 88% of reviewers are taking more than two hours on journal reviews. We also report on qualitative results. To write a good review, the important criteria were it should be factual and helpful, ranked above others such as being detailed or kind. The most important features of papers that result in positive reviews are clear and supported validation, an interesting problem, and novelty. Conversely, negative reviews tend to result from papers that have a mismatch between the method and the claims and from those with overly grandiose claims. The main recommendation for authors is to make the contribution of the work very clear in their paper. In addition, reviewers viewed data availability and its consistency as being important.

Software Engineering,Computers and Society
Peer Review Is Primary: Presentations, Publications, Promotions, and Practice

Kendall M Campbell,Edgar Figueroa,Donna Baluchi,José E Rodríguez

DOI: https://doi.org/10.22454/PRiMER.2024.148162

2024-08-05

PRiMER

Abstract:Peer review is primarily thought of as the process used to determine whether manuscripts are published in medical or other academic journals. While a publication may be one outcome of peer review, this article shares a model of 4 Ps to remind faculty of some important additional applications of peer review. The 4 Ps are publication, presentation, promotion, and practice. The medical literature offers few reasons why faculty should get involved in peer review. In this article, we define peer review, illustrate the role of peer review in four important processes, describe how the volume of material to review has changed over time, and share how participation in these processes promotes career advancement. Understanding the peer review process and its benefits can encourage professionals to participate in peer review in any of the four Ps as they recognize the benefits to their discipline and their career.
The Miracle of Peer Review and Development in Science: An Agent-Based Model

Simone Righi,Károly Takács

DOI: https://doi.org/10.48550/arXiv.1607.02574

2016-07-09

Abstract:It is not easy to rationalize how peer review, as the current grassroots of science, can work based on voluntary contributions of reviewers. There is no rationale to write impartial and thorough evaluations. Consequently, there is no risk in submitting low-quality work by authors. As a result, scientists face a social dilemma: if everyone acts according to his or her own self-interest, low scientific quality is produced. Still, in practice, reviewers as well as authors invest high effort in reviews and submissions. We examine how the increased relevance of public good benefits (journal impact factor), the editorial policy of handling incoming reviews, and the acceptance decisions that take into account reputational information can help the evolution of high-quality contributions from authors. High effort from the side of reviewers is problematic even if authors cooperate: reviewers are still best off by producing low-quality reviews, which does not hinder scientific development, just adds random noise and unnecessary costs to it. We show with agent-based simulations that tacit agreements between authors that are based on reciprocity might decrease these costs, but does not result in superior scientific quality. Our study underlines why certain self-emerged current practices, such as the increased importance of journal metrics, the reputation-based selection of reviewers, and the reputation bias in acceptance work efficiently for scientific development. Our results find no answers, however, how the system of peer review with impartial and thorough evaluations could be sustainable jointly with rapid scientific development.

Physics and Society,Digital Libraries,Multiagent Systems

Peer review demystified: part 1

Peer review demystified: part 2

The Peer Review Process: Past, Present, and Future

Scientific peer review in the modern era: A comprehensive guide

An overview of the peer review process in biomedical sciences

The Peer Review Process

Motivating and Maintaining Ethics, Equity, Effectiveness, Efficiency, and Expertise in Peer Review

How to be a good reviewer: A step-by-step guide for approaching peer review of a scientific manuscript

Peer Review and Medical Journals

Mastering the scientific peer review process: tips for young authors from a young senior editor

The Importance and Responsibilities of Reviewers

How I am working as a reviewer?

A history and development of peer-review process

Avoiding a Tragedy of the Commons in the Peer Review Process

The delicate nature of a constructive peer review: pearls from the editorial board

Knowledge behind the scenes of a peer-reviewed journal helps in publishing your manuscript

Understanding Peer Review of Software Engineering Papers

Peer Review Is Primary: Presentations, Publications, Promotions, and Practice

The Miracle of Peer Review and Development in Science: An Agent-Based Model