Promoting research quality

Ulrich Dirnagl,Nonia Pariente

DOI: https://doi.org/10.1371/journal.pbio.3002554

IF: 9.8

2024-02-29

PLoS Biology

Abstract:Plenty of awards recognize scientific contributions, but a unique and important one honors those whose efforts significantly enhance the quality and robustness of research. We discuss why this is important to promote trust in science.

biochemistry & molecular biology,biology

What problem does this paper attempt to address?

Towards responsible research assessment: How to reward research quality

Anne Gärtner,Daniel Leising,Felix D Schönbrodt,Felix D. Schönbrodt

DOI: https://doi.org/10.1371/journal.pbio.3002553

IF: 9.8

2024-02-27

PLoS Biology

Abstract:Researchers would be more willing to prioritize research quality over quantity if the incentive structure of the academic system aligned with this goal. The winner of a 2023 Einstein Foundation Award for Promoting Quality in Research explains how they rose to this challenge.

biochemistry & molecular biology,biology
Promoting and Supporting Innovation Through Collaborative Research

Jim Gentile

DOI: https://doi.org/10.1096/fasebj.26.1_supplement.617.1

2012-04-01

The FASEB Journal

Abstract:Many of the major scientific challenges of the 21st century are likely to require collaborative and interdisciplinary teams. Creating good “team players” who can work synergistically to undertake high‐risk, high‐reward research should be a priority, and demonstrating success in collaborations should be encouraged by private and federal granting agencies, and more highly rewarded by universities in tenure and promotion decisions. Given the challenges that our nation faces, tenure and promotion decisions should no longer be based merely on the number of papers published or overhead dollars generated. Journal editorial boards and foundation peer‐review panels are essential to the highest‐quality science, but they should not be viewed as surrogate members of tenure and promotion committees. Rewarding careers paths should be clearly visible to the nation's newly minted science Ph.D.s, postdoctoral researchers and “tenure track” professors. Allowing these early‐career scientists more expansive opportunities during their most creative years would make the United States, and the academic communities in which they serve, an increasingly more vibrant and creative environments. The heart and soul of American science and enterprise is the willingness to take risks, to learn from failure and to repeat the process until success is achieved. Our early‐career scientists should have the same opportunity.

cell biology,biochemistry & molecular biology,biology
Promoting an open research culture

B. A. Nosek,G. Alter,G. C. Banks,D. Borsboom,S. D. Bowman,S. J. Breckler,S. Buck,C. D. Chambers,G. Chin,G. Christensen,M. Contestabile,A. Dafoe,E. Eich,J. Freese,R. Glennerster,D. Goroff,D. P. Green,B. Hesse,M. Humphreys,J. Ishiyama,D. Karlan,A. Kraut,A. Lupia,P. Mabry,T. Madon,N. Malhotra,E. Mayo-Wilson,M. McNutt,E. Miguel,E. Levy Paluck,U. Simonsohn,C. Soderberg,B. A. Spellman,J. Turitto,G. VandenBos,S. Vazire,E. J. Wagenmakers,R. Wilson,T. Yarkoni

DOI: https://doi.org/10.1126/science.aab2374

IF: 56.9

2015-06-26

Science

Abstract:Author guidelines for journals could help to promote transparency, openness, and reproducibility

multidisciplinary sciences
What Is Quality in Research? Building a Framework of Design, Process and Impact Attributes and Evaluation Perspectives

Alessandro Margherita,Gianluca Elia,Claudio Petti

DOI: https://doi.org/10.3390/su14053034

IF: 3.9

2022-03-04

Sustainability

Abstract:The strategic relevance of innovation and scientific research has amplified the attention towards the definition of quality in research practice. However, despite the proliferation of evaluation metrics and procedures, there is a need to go beyond bibliometric approaches and to identify, more explicitly, what constitutes good research and which are its driving factors or determinants. This article reviews specialized research policy, science policy and scientometrics literature to extract critical dimensions associated with research quality as presented in a vast although fragmented theory background. A literature-derived framework of research quality attributes is, thus, obtained, which is subject to an expert feedback process, involving scholars and practitioners in the fields of research policy and evaluation. The results are represented by a structured taxonomy of 66 quality attributes providing a systemic definition of research quality. The attributes are aggregated into a three-dimensional framework encompassing research design (ex ante), research process (in-process) and research impact (ex post) perspectives. The main value of the study is to propose a literature-derived and comprehensive inventory of quality attributes and perspectives of evaluation. The findings can support further theoretical developments and research policy discussions on the ultimate drivers of quality and impact of scientific research. The framework can be also useful to design new exercises or procedures of research evaluation based on a multidimensional view of quality.

environmental sciences,environmental studies,green & sustainable science & technology
Exploring COVID-19 research credibility among Spanish scientists

Eduardo Garcia-Garzon,Ariadna Angulo-Brunet,Oscar Lecuona,Juan Ramón Barrada,Guido Corradi

DOI: https://doi.org/10.1007/s12144-022-02797-6

IF: 2.8

2022-02-28

Current Psychology

Abstract:Amidst a worldwide vaccination campaign, trust in science plays a significant role when addressing the COVID-19 pandemic. Given current concerns regarding research standards, we were interested in how Spanish scholars perceived COVID-19 research and the extent to which questionable research practices and potentially problematic academic incentives are commonplace. We asked researchers to evaluate the expected quality of their COVID-19 projects and other peers’ research and compared these assessments with those from scholars not involved in COVID-19 research. We investigated self-admitting and estimated rates of questionable research practices and attitudes towards current research status. Responses from 131 researchers suggested that COVID-19 evaluations followed partisan lines, with scholars being more pessimistic about others’ colleagues’ research than their own. Additionally,researchers not involved in COVID-19 projects were more negative than their participating peers. These differences were particularly notable for areas such as the expected theoretical foundations or overall quality of the research, among others. Most Spanish scholars expected questionable research practices and inadequate incentives to be widespread. In these two aspects, researchers tended to agree regardless of their involvement in COVID-19 research. We provide specific recommendations for improving future meta-science studies, such as redefining QRPs as inadequate research practices (IRP). This change could help avoid key controversies regarding QRPs’ definition while highlighting their detrimental impact. Lastly, we join previous calls to improve transparency and academic career incentives as a cornerstone for generating trust in science.

psychology, multidisciplinary
Leading for research excellence and integrity: A qualitative investigation of the relationship-building practices of exemplary principal investigators

Alison L. Antes,Ashley Kuykendall,James M. DuBois

DOI: https://doi.org/10.1080/08989621.2019.1611429

2019-04-03

Accountability in Research

Abstract:We conducted semi-structured, telephone interviews with 52 federally funded researchers nominated as exemplars for their integrity and professional conduct, and their scientific achievements. The aim was to identify the practices they report utilizing to build respectful relationships in their teams. We found four practices, holding meetings, providing supervision and guidance, encouraging shared ownership, and expressing values, which were also important to performing high-quality, compliant research, were essential to fostering relationships. The most common practice described for building relationships was actively and deliberately cultivating a positive team environment. Additionally, exemplars described the need to lead by example, tailor their approach to the needs of individuals, address interpersonal conflict, and hire team members cautiously. We also identified practices the exemplars reported as important to managing the demands of their work and found that encouraging shared ownership and tailoring to individuals supported this goal. Additional strategies related to prioritization and planning, seeking advice, engaging in self-care, and managing emotional reactions. Finally, we identified priorities guiding the exemplars' practices. Key priorities included providing outstanding mentoring, building collaborations and relationships, and engaging in discovery and innovation. Investigators require exceptional leadership skills but receive limited systematic leadership training. Addressing this gap would advance research excellence and integrity.
"Best Paper" awards lack transparency, inclusivity, and support for Open Science

Malgorzata Lagisz,Joanna Rutkowska,Upama Aich,Robert M Ross,Manuela S Santana,Joshua Wang,Nina Trubanová,Matthew J Page,Andrew Adrian Yu Pua,Yefeng Yang,Bawan Amin,April Robin Martinig,Adrian Barnett,Aswathi Surendran,Ju Zhang,David N Borg,Jafsia Elisee,James G Wrightson,Shinichi Nakagawa

DOI: https://doi.org/10.1371/journal.pbio.3002715

2024-07-23

Abstract:Awards can propel academic careers. They also reflect the culture and values of the scientific community. But do awards incentivize greater transparency, inclusivity, and openness in science? Our cross-disciplinary survey of 222 awards for the "best" journal articles across all 27 SCImago subject areas revealed that journals and learned societies administering such awards generally publish little detail on their procedures and criteria. Award descriptions were brief, rarely including contact details or information on the nominations pool. Nominations of underrepresented groups were not explicitly encouraged, and concepts that align with Open Science were almost absent from the assessment criteria. At the same time, 10% of awards, especially the recently established ones, tended to use article-level impact metrics. USA-affiliated researchers dominated the winner's pool (48%), while researchers from the Global South were uncommon (11%). Sixty-one percent of individual winners were men. Overall, Best Paper awards miss the global calls for greater transparency and equitable access to academic recognition. We provide concrete and implementable recommendations for scientific awards to improve the scientific recognition system and incentives for better scientific practice.
Impact Vitality - A Measure for Excellent Scientists

Nadine Rons,Lucy Amez

DOI: https://doi.org/10.48550/arXiv.1307.6770

2013-07-25

Digital Libraries

Abstract:In many countries and at European level, research policy increasingly focuses on 'excellent' researchers. The concept of excellence however is complex and multidimensional. For individual scholars it involves talents for innovative knowledge creation and successful transmission to peers, as well as management capacities. Excellence is also a comparative concept, implying the ability to surpass others [TIJSSEN, 2003]. Grants are in general awarded based on assessments by expert committees. While peer review is a widely accepted practice, it nevertheless is also subject to criticism. At higher aggregation levels, peer assessments are often supported by quantitative measures. At individual level, most of these measures are much less appropriate and there is a need for new, dedicated indicators.
Improving Systems to Promote Research Integrity

Leilani B Mercado-Asis,Ma. Lourdes D Maglinao

DOI: https://doi.org/10.35460/2546-1621.2022-sp05

2022-12-07

Abstract:Research integrity is manifested thru the use of honest and verifiable research methods with adherence to accepted professional codes. Recently, trustworthiness in research has been challenged by various forms of research misconduct, such as analytical flexibility, data dredging, HARKing (hypothesis after research knowledge), plagiarism, and selective and distorted reporting. Drivers of research misconduct have been identified as institutional--publication incentives to pursue a career, researcher--metric of success is publication volume, and the journal-- more likely to accept papers with positive. The open-access mode propelling the proliferation of predatory journals is causing a dilemma to sound research reporting. Measures were established to curtail research integrity challenges, such as study registration, open data, common reporting standards, a team of rivals, and blind analysis. This report will elaborate and provide insight into what influenced research misconduct, how it can be mitigated, and how to maintain a credible research environment.
The Importance of Survey Research Standards

Jack E. Fincham,JoLaine R. Draugalis

DOI: https://doi.org/10.5688/ajpe7714

IF: 2.876

2013-02-12

American Journal of Pharmaceutical Education

Abstract:Every discipline within fields of research has instituted guidelines and templates for research endeavors and subsequent publications of findings, with the ultimate result being an increase in quality and acceptance by researchers within and across disciplines. These significant efforts are by nature ongoing, as well they should. These enhancements and guideline developments have been instituted in basic science disciplines, clinical pharmacy, and pharmacy administration relevant and related to subsequent scholarly publication of research findings. Specific research endeavors have included bench research, clinical trials and randomized clinical trials, meta analyses, outcomes research, and large scale database analyses. A similar need for quality and standardization also exists for survey research and scholarship. The purpose of this paper is to clarify why this is important and crucial for the Journal and our academy.

education, scientific disciplines,pharmacology & pharmacy
On choosing meaningful research projects in the natural sciences

Philip G Judge,Isabel Lipartito,Roberto Casini

DOI: https://doi.org/10.48550/arXiv.1711.09068

2017-11-24

Physics Education

Abstract:Over the last few decades, the nature of scientific research has changed in response to external influences. Firstly, powerful networked computers have become a standard tool. Secondly, society presses ever harder for research to deliver something "useful" back to society, both through the kinds of funding opportunities that are made available, and through a critical public eye. Many funding agencies now demand "deliverables" that seem to select research of a particular kind. Lastly, teamwork, often within very large projects, has become commonplace. Here, we step back and ask how prospective research scientists might select productive research projects in this evolving environment. We hope that our suggestions might also help to improve public understanding and thereby restore flagging faith in science.
Protecting and promoting editorial independence

Kamaldeep Bhui,Aileen O'Brien,Rachel Upthegrove,Alexander C. Tsai,Mustafa Soomro,Giles Newton-Howes,Matthew R. Broome,Andrew Forrester,Patricia Casey,Anne M. Doherty,William Lee,Kenneth R. Kaufman

DOI: https://doi.org/10.1192/bjp.2024.6

2024-02-16

The British Journal of Psychiatry

Abstract:Summary We argue that editorial independence, through robust practice of publication ethics and research integrity, promotes good science and prevents bad science. We elucidate the concept of research integrity, and then discuss the dimensions of editorial independence. Best practice guidelines exist, but compliance with these guidelines varies. Therefore, we make recommendations for protecting and strengthening editorial independence.

psychiatry
Editors' Responsibility for Publishing High-Quality Research Results: A Worldwide Study Into Current Challenges in Quality Assessment Processes

Katarina Krapež

DOI: https://doi.org/10.18690/lexonomica.14.1.127-152.2022

2022-06-22

Lexonomica

Abstract:The contemporary scientific community faces significant challenges in assuring research credibility. Since the onset of the COVID-19 pandemic, there has been an even greater interest in swift publications of valid research results, and individual editors of reputable academic journals might feel pressured to ensure high quality of published papers. Aiming to explore how editors perceive their management of the quality assessment processes and to identify decisions and processes which editors believe to be the most critical for securing high-quality publications, we conducted a qualitative survey. 258 editors of scientific journals from 42 countries participated in the study. Key findings of the first part of the study show that across disciplines, editors perceived the manuscripts as excellent if they were innovative, scientifically sound, well-written and well-argued, addressed a significant topic, contained useful results, and had the potential to change (improve) something. Secondly, originality emerged as the leading quality criteria in manuscript quality assessment, followed by validity and significance. Finally, results indicate that factors influencing the overall complexity of editorial quality assessment are: (i) clearly defined minimum threshold of required quality, (ii) consistency of assessment between individual quality criteria, and (iii) experience of the editor.
Science Needs You: Mobilizing for Diversity in Award Recognition

Elizabeth Novoa-Monsalve,David Patterson,Stephanie Ludi,Daniel E. Acuna

DOI: https://doi.org/10.1145/3651150

IF: 22.7

2024-08-02

Communications of the ACM

Abstract:Insights on how to create a more inclusive and equitable system of award recognition in the scientific community.

computer science, theory & methods, software engineering, hardware & architecture
Earning respect and trust

H Holden Thorp,H. Holden Thorp

DOI: https://doi.org/10.1126/science.ado3040

IF: 56.9

2024-02-02

Science

Abstract:Respect for, and trust in, science may be at an all-time low. In the United States, a 2023 Pew Research poll showed that only 57% of the population believed science has had a positive impact on society, and a Gallup poll showed that confidence in higher education was down to 36%. If the Gallup poll were done now, support would likely be even lower, given recent events with university presidents, from questions about their research integrity to their explanations for policies on student speech. I'm frequently asked what can be done about all of this, especially in the realm of science. Many scientists think the challenge has largely to do with science communication, which is certainly important. But first, the scientific community must begin to conduct itself in the same manner that it is asking of the public, and that means treating everyone in the scientific community with respect.

multidisciplinary sciences
Navigating the Science System: Research Integrity and Academic Survival Strategies

Andrea Reyes Elizondo,Wolfgang Kaltenbrunner

DOI: https://doi.org/10.1007/s11948-024-00467-3

2024-04-03

Abstract:Research Integrity (RI) is high on the agenda of both institutions and science policy. The European Union as well as national ministries of science have launched ambitious initiatives to combat misconduct and breaches of research integrity. Often, such initiatives entail attempts to regulate scientific behavior through guidelines that institutions and academic communities can use to more easily identify and deal with cases of misconduct. Rather than framing misconduct as a result of an information deficit, we instead conceptualize Questionable Research Practices (QRPs) as attempts by researchers to reconcile epistemic and social forms of uncertainty in knowledge production. Drawing on previous literature, we define epistemic uncertainty as the inherent intellectual unpredictability of scientific inquiry, while social uncertainty arises from the human-made conditions for scientific work. Our core argument-developed on the basis of 30 focus group interviews with researchers across different fields and European countries-is that breaches of research integrity can be understood as attempts to loosen overly tight coupling between the two forms of uncertainty. Our analytical approach is not meant to relativize or excuse misconduct, but rather to offer a more fine-grained perspective on what exactly it is that researchers want to accomplish by engaging in it. Based on the analysis, we conclude by proposing some concrete ways in which institutions and academic communities could try to reconcile epistemic and social uncertainties on a more collective level, thereby reducing incentives for researchers to engage in misconduct.
Five creative ways to promote reproducible science

Josefina Weinerova,Rotem Botvinik-Nezer,Roni Tibon

DOI: https://doi.org/10.1038/s41562-023-01808-1

IF: 24.252

2024-01-29

Nature Human Behaviour

Abstract:The importance of reproducible scientific practices is widely acknowledged. However, limited resources and lack of external incentives have hindered their adoption. Here, we explore ways to promote reproducible science in practice.

psychology, experimental,neurosciences,multidisciplinary sciences
Enhancing Research Methodology and Academic Publishing: A Structured Framework for Quality and Integrity

Md. Jalil Piran,Nguyen H. Tran

2024-12-12

Abstract:Following a brief introduction to research, research processes, research types, papers, reviews, and evaluations, this paper presents a structured framework for addressing inconsistencies in research methodology, technical writing, quality assessment, and publication standards across academic disciplines. Using a four-dimensional evaluation model that focuses on 1) technical content, 2) structural coherence, 3) writing precision, and 4) ethical integrity, this framework not only standardizes review and publication processes but also serves as a practical guide for authors in preparing high-quality manuscripts. Each of these four dimensions cannot be compromised for the sake of another. Following that, we discuss the components of a research paper adhering to the four-dimensional evaluation model in detail by providing guidelines and principles. By aligning manuscripts with journal standards, reducing review bias, and enhancing transparency, the framework contributes to more reliable and reproducible research results. Moreover, by strengthening cross-disciplinary credibility, improving publication consistency, and fostering public trust in academic literature, this initiative is expected to positively influence both research quality and scholarly publishing's reputation.

Digital Libraries
Nurturing a positive research culture within your laboratory

Adrian Liston,Denise C Fitzgerald

DOI: https://doi.org/10.12688/wellcomeopenres.22492.1

2024-10-15

Abstract:As a Principal Investigator leading a research team, creating a positive research culture for your team members is one of the best long-term investments you can make, for your research programme, for the sector and for society. A positive research environment is one where team members are empowered, recognised, have a clear career developmental pathway, can contribute to impactful and reproducible research and, ideally, propagate these effects. While these virtues can arise organically from a team built around kindness and integrity, they are also values that should be deliberately embedded within your lab. Here we provide advice on how to create a culture of integrity and a culture of belonging for your team members. We focus on thoughtful consideration of your key lab values, and the use of structure, language and your personal actions to make these values explicit. A holistic approach to integrating positive culture throughout every facet of your research team creates a system that can be self-sustaining in scientific integrity and more resilient to negative challenges. Starting on the pathway to self-improvement as a manager, recognising that this requires often uncomfortable self-reflection, provides both personal and professional reward.
Über wissenschaftliche Exzellenz und Wettbewerb

Gerald Schweiger

2023-10-18

Abstract:The pursuit of excellence seems to be the True North of academia. What is meant by excellence? Can excellence be measured? This article discusses the concept of excellence in the context of research and competition.

General Economics

Promoting research quality

Towards responsible research assessment: How to reward research quality

Promoting and Supporting Innovation Through Collaborative Research

Promoting an open research culture

What Is Quality in Research? Building a Framework of Design, Process and Impact Attributes and Evaluation Perspectives

Exploring COVID-19 research credibility among Spanish scientists

Leading for research excellence and integrity: A qualitative investigation of the relationship-building practices of exemplary principal investigators

"Best Paper" awards lack transparency, inclusivity, and support for Open Science

Impact Vitality - A Measure for Excellent Scientists

Improving Systems to Promote Research Integrity

The Importance of Survey Research Standards

On choosing meaningful research projects in the natural sciences

Protecting and promoting editorial independence

Editors' Responsibility for Publishing High-Quality Research Results: A Worldwide Study Into Current Challenges in Quality Assessment Processes

Science Needs You: Mobilizing for Diversity in Award Recognition

Earning respect and trust

Navigating the Science System: Research Integrity and Academic Survival Strategies

Five creative ways to promote reproducible science

Enhancing Research Methodology and Academic Publishing: A Structured Framework for Quality and Integrity

Nurturing a positive research culture within your laboratory

Über wissenschaftliche Exzellenz und Wettbewerb