Update to the Journal of Water Resources Planning and Management's Reproducible Results Policy

James H. Stagge,David E. Rosenberg,Anthony M. Castronova,Avi Ostfeld,Amber Spackman Jones

DOI: https://doi.org/10.1061/jwrmd5.wreng-6726

IF: 3.457

2024-10-30

Journal of Water Resources Planning and Management

Abstract:Journal of Water Resources Planning and Management, Volume 151, Issue 1, January 1, 2025.

water resources,engineering, civil

What problem does this paper attempt to address?

Journal of Water Resources Planning and Management's Reproducibility Review Program: Accomplishments, Lessons, and Next Steps

James H. Stagge,David E. Rosenberg,Anthony M. Castronova,Avi Ostfeld,Amber Spackman Jones

DOI: https://doi.org/10.1061/jwrmd5.wreng-6559

IF: 3.457

2024-05-18

Journal of Water Resources Planning and Management

Abstract:Journal of Water Resources Planning and Management, Volume 150, Issue 8, August 1, 2024.

water resources,engineering, civil
Assessing data availability and research reproducibility in hydrology and water resources

James H. Stagge,David E. Rosenberg,Adel M. Abdallah,Hadia Akbar,Nour A. Attallah,Ryan James

DOI: https://doi.org/10.1038/sdata.2019.30

2019-02-26

Scientific Data

Abstract:There is broad interest to improve the reproducibility of published research. We developed a survey tool to assess the availability of digital research artifacts published alongside peer-reviewed journal articles (e.g. data, models, code, directions for use) and reproducibility of article results. We used the tool to assess 360 of the 1,989 articles published by six hydrology and water resources journals in 2017. Like studies from other fields, we reproduced results for only a small fraction of articles (1.6% of tested articles) using their available artifacts. We estimated, with 95% confidence, that results might be reproduced for only 0.6% to 6.8% of all 1,989 articles. Unlike prior studies, the survey tool identified key bottlenecks to making work more reproducible. Bottlenecks include: only some digital artifacts available (44% of articles), no directions (89%), or all artifacts available but results not reproducible (5%). The tool (or extensions) can help authors, journals, funders, and institutions to self-assess manuscripts, provide feedback to improve reproducibility, and recognize and reward reproducible articles as examples for others.

multidisciplinary sciences
National-Scale Optimized Design of Cost-Effective Water Supply and Transfer Systems

James Tomlinson,Andrew Slaughter,Evgenii Matrosov,Jonny Wilson,Richard Blackwell,Tracey Dunford,Margaret Read,Jonathan Dennis,Julien J. Harou

DOI: https://doi.org/10.1061/jwrmd5.wreng-6506

IF: 3.457

2024-10-25

Journal of Water Resources Planning and Management

Abstract:Journal of Water Resources Planning and Management, Volume 151, Issue 1, January 1, 2025.

water resources,engineering, civil
Repeatability, Reproducibility, Replicability, Reusability (4R) in Journals' Policies and Software/Data Management in Scientific Publications: A Survey, Discussion, and Perspectives

José Armando Hernández,Miguel Colom

2023-12-18

Abstract:With the recognized crisis of credibility in scientific research, there is a growth of reproducibility studies in computer science, and although existing surveys have reviewed reproducibility from various perspectives, especially very specific technological issues, they do not address the author-publisher relationship in the publication of reproducible computational scientific articles. This aspect requires significant attention because it is the basis for reliable research. We have found a large gap between the reproducibility-oriented practices, journal policies, recommendations, publisher artifact Description/Evaluation guidelines, submission guides, technological reproducibility evolution, and its effective adoption to contribute to tackling the crisis. We conducted a narrative survey, a comprehensive overview and discussion identifying the mutual efforts required from Authors, Journals, and Technological actors to achieve reproducibility research. The relationship between authors and scientific journals in their mutual efforts to jointly improve the reproducibility of scientific results is analyzed. Eventually, we propose recommendations for the journal policies, as well as a unified and standardized Reproducibility Guide for the submission of scientific articles for authors. The main objective of this work is to analyze the implementation and experiences of reproducibility policies, techniques and technologies, standards, methodologies, software, and data management tools required for scientific reproducible publications. Also, the benefits and drawbacks of such an adoption, as well as open challenges and promising trends, to propose possible strategies and efforts to mitigate the identified gaps. To this purpose, we analyzed 200 scientific articles, surveyed 16 Computer Science journals, and systematically classified them according to reproducibility strategies, technologies, policies, code citation, and editorial business. We conclude there is still a reproducibility gap in scientific publications, although at the same time also the opportunity to reduce this gap with the joint effort of authors, publishers, and technological providers.

Software Engineering
Partnering With Authors to Enhance Reproducibility at JASA

Julia Wrobel,Emily C. Hector,Lorin Crawford,Lucy D’Agostino McGowan,Natalia da Silva,Jeff Goldsmith,Stephanie Hicks,Michael Kane,Youjin Lee,Vinicius Mayrink,Christopher J. Paciorek,Therri Usher,Julian Wolfson

DOI: https://doi.org/10.1080/01621459.2024.2340557

IF: 4.369

2024-04-27

Journal of the American Statistical Association

Abstract:In 2016, JASA Applications and Case Studies (ACS) introduced a reproducibility initiative to address the lack of standardized practices for reproducibility in scientific research. This initiative established minimum criteria for the inclusion of code, data, and workflow at JASA ACS, and piloted a new editorial role, Associate Editor of Reproducibility (AER), to implement these standards. This initiative has since expanded from ACS to all original research manuscripts published at JASA , including those submitted to JASA Theory and Methods (TM). Since the inception of the JASA reproducibility initiative, the team of AERs has grown, the process has become more standardized, the role of an AER in guiding reproducibility in statistical research has been refined (Willis and Stodden Citation 2020 ) and, in 2023, we implemented a reproducibility award 1 to recognize papers published in JASA with outstanding reproducibility materials. The goals of this editorial are to (a) explain the type of reproducibility that is being addressed during the review process at JASA , (b) describe the steps of this process and its underlying philosophy, and (c) clarify how authors can streamline the review of their reproducibility materials.

statistics & probability
Erratum for "The Role of Reservoir Reoperation to Mitigate Climate Change Impacts on Hydropower and Environmental Water Demands"

Ana Paula Dalcin,João Paulo Lyra Fialho Brêda,Guilherme Fernandes Marques,Amaury Tilmant,Rodrigo Cauduro Dias de Paiva,Paulo Yoshio Kubota

DOI: https://doi.org/10.1061/jwrmd5.wreng-6743

IF: 3.457

2024-09-15

Journal of Water Resources Planning and Management

Abstract:Journal of Water Resources Planning and Management, Volume 150, Issue 11, November 1, 2024.

water resources,engineering, civil
The Basic Problem of Contemporary Water Resources Environment Research: Translation and Analysis from The English Practice Report on Water Resources

Yingjie Qu,Hui Liu

DOI: https://doi.org/10.2112/jcr-si104-019.1

2020-10-05

Journal of Coastal Research

Abstract:Qu, Y.J. and Liu, H., 2020. The basic problem of contemporary water resources environment research: translation and analysis from the English practice report on water resources. In: Guido Aldana, P.A. and Kantamaneni, K. (eds.), Advances in Water Resources, Coastal Management, and Marine Science Technology. Journal of Coastal Research, Special Issue No. 104, pp. 106110. Coconut Creek (Florida), ISSN 0749-0208. Water resources have become an important resource for people's life and production, and with the rapid development of modern economy and society, the problem of water pollution and shortage is becoming more and more serious. China has a large land and a large population, and the amount of water resources per capita is difficult to meet people's real needs. Therefore, combined with the importance of water resources, strengthen the basic problems of environmental research on water resources, not only to strengthen the use and development of water resources, but also to do a good job of sewage, water conservation environmental protection strategies. To achieve the ecological environment and economic development can be balanced and healthy development. With the concept of scientific, rational and sustainable development, the research and analysis of water resources environmental protection should be carried out, so that the specific details of the work can be put into practice, so as to effectively protect the water resources environment and realize the maximum benefit of water resources. The study of water resources environment in the English practice report on water resources also clearly proposes to solve its basic problems.

geosciences, multidisciplinary,environmental sciences,geography, physical
Reshaping the publication process: Addiction Research and Theory joins Peer Community In Registered Reports

Charlotte R. Pennington,Derek Heim

DOI: https://doi.org/10.1080/16066359.2021.1931142

2021-06-01

Abstract:There is growing awareness of reproducibility concerns within the scientific literature; whilst journals continue to be filled with seemingly beautiful results, teams of researchers are lifting the lid on the veracity of scientific claims and the replicability of findings in a growing number of disciplines. Highlighting the global scale of this issue, the Open Science Collaboration (<a href="#">2015</a>) performed direct replications of 100 psychological research studies and found that whilst 97% of the original studies reported significant results, this declined to 36% for replications of these. Moreover, the average effect size of the replications was half the magnitude of the average across original studies. Several human decision-making factors have been proposed to underlie irreproducibility, such as selective reporting and questionable research practices (QRPs; Simmons et al. <a href="#">2011</a>; John et al. <a href="#">2012</a>) fueled by pressures to achieve novel, surprising results and incentive structures that reward quantity over quality (Giner-Sorolla <a href="#">2012</a>; Frith <a href="#">2020</a>; Munafò et al. <a href="#">2020</a>). As can be seen in <a href="#F0001">Figure 1</a>, there are many threats that can arise at each stage of the hypothetico-deductive scientific process, which can undermine the robustness of published research and constrain knowledge accumulation. In their 'manifesto for reproducible science', Munafò et al. (<a href="#">2017</a>) propose five measures that target these threats with the goal of improving research: (1) methods, (2) reporting and dissemination, (3) reproducibility, (4) evaluation, and (5) incentives. Adoption of open science practices will allow many of these measures to be achieved and calls for their implementation in addiction science are becoming louder (Heirene <a href="#">2021</a>; Louderback et al. <a href="#">2021</a>; Pennington et al. <a href="#">2021</a>). However, to be successful, academic journals also need to adapt and recentre their focus on scientific rigor rather than the metamorphosis of study results. In this Editorial, we introduce two types of Registered Reports (RRs) that satisfy all these measures and highlight our commitment by offering these publishing tracks to authors through ART. We first outline the 'traditional' RR format (which ART has offered since 2019) and its associated advantages, followed by the introduction of a new initiative launched in April 2021 – Peer Community In Registered Reports (PCI RR).

substance abuse,social issues
Balancing Effectiveness and Equity in Sustainable Water Management Transitions: Case of the Miami-Dade Water and Sewer Department

Koorosh Azizi,Jesse L. Barnes,Aaron Deslatte,Elizabeth A. Koebele,John M. Anderies,Margaret Garcia

DOI: https://doi.org/10.1061/jwrmd5.wreng-6523

IF: 3.457

2024-10-19

Journal of Water Resources Planning and Management

Abstract:Journal of Water Resources Planning and Management, Volume 151, Issue 1, January 1, 2025.

water resources,engineering, civil
Erratum for the Research Article: “The intensification of the water footprint of hydraulic fracturing” by A. J. Kondash, N. E. Lauer and A. Vengosh

DOI: https://doi.org/10.1126/sciadv.aax8764

IF: 13.6

2019-05-03

Science Advances

Abstract:[This corrects the article DOI: 10.1126/sciadv.aar5982.].

multidisciplinary sciences
Raising the bar for reproducible science at the U.S. Environmental Protection Agency Office of Research and Development

Barbara Jane George,Jon R Sobus,Lara P Phelps,Brenda Rashleigh,Jane Ellen Simmons,Ronald N Hines,Community of Practice for Statistics Guidance Documents Working Groups,Diana R Hall,Michael D Hays,Roxanne Johnson,Martin Blake Phillips,Najwa Haykal-Coates,James L Crooks,Andrew J R Gillespie,Janet A Nestlerode,Alan Vette,Timothy J Wade,Rory B Conolly,Kristen M Foley,Elaina M Kenyon,Susan Yee

DOI: https://doi.org/10.1093/toxsci/kfv020

Abstract:Considerable concern has been raised regarding research reproducibility both within and outside the scientific community. Several factors possibly contribute to a lack of reproducibility, including a failure to adequately employ statistical considerations during study design, bias in sample selection or subject recruitment, errors in developing data inclusion/exclusion criteria, and flawed statistical analysis. To address some of these issues, several publishers have developed checklists that authors must complete. Others have either enhanced statistical expertise on existing editorial boards, or formed distinct statistics editorial boards. Although the U.S. Environmental Protection Agency, Office of Research and Development, already has a strong Quality Assurance Program, an initiative was undertaken to further strengthen statistics consideration and other factors in study design and also to ensure these same factors are evaluated during the review and approval of study protocols. To raise awareness of the importance of statistical issues and provide a forum for robust discussion, a Community of Practice for Statistics was formed in January 2014. In addition, three working groups were established to develop a series of questions or criteria that should be considered when designing or reviewing experimental, observational, or modeling focused research. This article describes the process used to develop these study design guidance documents, their contents, how they are being employed by the Agency's research enterprise, and expected benefits to Agency science. The process and guidance documents presented here may be of utility for any research enterprise interested in enhancing the reproducibility of its science.
Influence of Subseasonal-to-Annual Water Supply Forecasts on Many-Objective Water System Robustness under Long-Term Change

Kyla Semmendinger,Scott Steinschneider

DOI: https://doi.org/10.1061/jwrmd5.wreng-6205

IF: 3.457

2024-02-24

Journal of Water Resources Planning and Management

Abstract:Journal of Water Resources Planning and Management, Volume 150, Issue 5, May 1, 2024.

water resources,engineering, civil
The past, present and future of Registered Reports

Christopher D. Chambers,Loukia Tzavella

DOI: https://doi.org/10.1038/s41562-021-01193-7

IF: 24.252

2021-11-15

Nature Human Behaviour

Abstract:Registered Reports are a form of empirical publication in which study proposals are peer reviewed and pre-accepted before research is undertaken. By deciding which articles are published based on the question, theory and methods, Registered Reports offer a remedy for a range of reporting and publication biases. Here, we reflect on the history, progress and future prospects of the Registered Reports initiative and offer practical guidance for authors, reviewers and editors. We review early evidence that Registered Reports are working as intended, while at the same time acknowledging that they are not a universal solution for irreproducibility. We also consider how the policies and practices surrounding Registered Reports are changing, or must change in the future, to address limitations and adapt to new challenges. We conclude that Registered Reports are promoting reproducibility, transparency and self-correction across disciplines and may help reshape how society evaluates research and researchers.

psychology, experimental,neurosciences,multidisciplinary sciences
Outstanding Reviewers for Environmental Science: Water Research & Technology in 2018

Caitlyn Butler,Neal Tai Shung Chung,Pei‐Ying Hong,Kyoung-Yeol Kim,James Landon,Long D. Nghiem,Zhiyong Jason Ren,Adam J. de Smith,Zhiwei Wang,Qian Zhang

DOI: https://doi.org/10.1039/c9ew90025j

2019-01-01

Environmental Science Water Research & Technology

Abstract:Environ. Sci.: Water Res. Technol., 2019, 5, 1036 DOI: 10.1039/C9EW90025J
Watershed Science: Coupling hydrological science and water resources management

Chansheng He,Carol P. Harden,Joseph Holden,Anna Mdee

DOI: https://doi.org/10.1002/hyp.14889

IF: 3.784

2023-04-23

Hydrological Processes

Abstract:While there have been advancements in hydrological science and water resources management, the world continues to face a water‐supply crisis. In light of studies that report the missing links to be i) effective collaborations between researchers and practitioners and ii) interdisciplinary working, we promote the benefits of Watershed Science to address these gaps. Watershed Science incorporates basin hydrology, determines water resource thresholds, promotes water governance that forms partnerships between institutions, and enacts participatory decision making in water resources management. However, such partnerships must recognize the differential power and interests of different actors to ensure that outcomes are not skewed in favor of particular interests. It adds value to existing water management programs and has great potential for bringing disciplines and decision making together to address the global water crisis. This article is protected by copyright. All rights reserved.

water resources
Optimized suspect screening approach for a comprehensive assessment of the impact of best management practices in reducing micropollutants transport in the Potomac River watershed

Mary Grace E. Guardian,Ping He,Alysson Bermudez,Shuiwang Duan,Sujay S. Kaushal,Erik Rosenfeldt,Diana S. Aga

DOI: https://doi.org/10.1016/j.wroa.2021.100088

IF: 7.5

2021-05-01

Water Research X

Abstract:The Publisher regrets that this article is an accidental duplication of a published article,  http://dx.doi.org/10.1016/j.wroa.2021.100088. The duplicate article has therefore been withdrawn.The full Elsevier Policy on Article Withdrawal can be found at https://www.elsevier.com/about/our-business/policies/article-withdrawal.
Reflecting on a dynamic biennium: The Annals of Regional Science 2022–2023

Brian H. S. Kim,Martin Andersson,Janet Kohlhase

DOI: https://doi.org/10.1007/s00168-024-01262-5

2024-03-19

The Annals of Regional Science

Abstract:The 2024 editorial update of The Annals of Regional Science reaffirms the journal's dedication to transparency and scholarly excellence in the field of regional science. Despite the challenges posed by the pandemic, the journal has demonstrated resilience and continued to serve as a pivotal platform for scholarly discourse. This update highlights the journal's recent milestones, including changes to the editorial board with new members bringing fresh perspectives to drive the journal's mission forward. The editorial discusses the journal's performance, encompassing its impact factor and the diverse range of topics covered, emphasizing its role in advancing research aligned with Sustainable Development Goals, particularly SDG 8. The ARSC continues to evolve, embracing changes that reflect current trends and maintaining its influence in the academic community.

economics,environmental studies,regional & urban planning,geography
Sustainable computational science: the ReScience initiative

Nicolas P Rougier,Konrad Hinsen,Frédéric Alexandre,Thomas Arildsen,Lorena A Barba,Fabien C Y Benureau,C Titus Brown,Pierre de Buyl,Ozan Caglayan,Andrew P Davison,Marc-André Delsuc,Georgios Detorakis,Alexandra K Diem,Damien Drix,Pierre Enel,Benoît Girard,Olivia Guest,Matt G Hall,Rafael N Henriques,Xavier Hinaut,Kamil S Jaron,Mehdi Khamassi,Almar Klein,Tiina Manninen,Pietro Marchesi,Daniel McGlinn,Christoph Metzner,Owen Petchey,Hans Ekkehard Plesser,Timothée Poisot,Karthik Ram,Yoav Ram,Etienne Roesch,Cyrille Rossant,Vahid Rostami,Aaron Shifman,Jemma Stachelek,Marcel Stimberg,Frank Stollmeier,Federico Vaggi,Guillaume Viejo,Julien Vitay,Anya E Vostinar,Roman Yurchak,Tiziano Zito

DOI: https://doi.org/10.7717/peerj-cs.142

2017-12-18

Abstract:Computer science offers a large set of tools for prototyping, writing, running, testing, validating, sharing and reproducing results; however, computational science lags behind. In the best case, authors may provide their source code as a compressed archive and they may feel confident their research is reproducible. But this is not exactly true. James Buckheit and David Donoho proposed more than two decades ago that an article about computational results is advertising, not scholarship. The actual scholarship is the full software environment, code, and data that produced the result. This implies new workflows, in particular in peer-reviews. Existing journals have been slow to adapt: source codes are rarely requested and are hardly ever actually executed to check that they produce the results advertised in the article. ReScience is a peer-reviewed journal that targets computational research and encourages the explicit replication of already published research, promoting new and open-source implementations in order to ensure that the original research can be replicated from its description. To achieve this goal, the whole publishing chain is radically different from other traditional scientific journals. ReScience resides on GitHub where each new implementation of a computational study is made available together with comments, explanations, and software tests.
Registered Reports in the Journal of Sports Sciences

Grant Abt,Colin Boreham,Gareth Davison,Robin Jackson,Eric Wallace,A Mark Williams

DOI: https://doi.org/10.1080/02640414.2021.1950974

IF: 3.9428

2021-08-11

Journal of Sports Sciences

Abstract:We published a recent editorial on sample size estimation (Abt et al., 2020), which was one in a series of editorials over many years intended to encourage authors to improve the rigour of studies submitted to the Journal of Sports Sciences (Nevill et al., 2004; Winter et al., 2014). We continue this encouragement now with the introduction of the Registered Report submission format.

sport sciences
Registered reports and replications: An ongoing Journal of School Psychology initiative

Jeffery P Braden,Jeffery P. Braden

DOI: https://doi.org/10.1016/j.jsp.2024.101294

IF: 6.033

2024-04-01

Journal of School Psychology

Abstract:Recent psychological research suggests that many published studies cannot be replicated (e.g., Open Science Collaboration, 2015). The inability to replicate results suggests that there are influences and biases in the publication process that encourage publication of unusual-rather than representative-results, and that also discourage independent replication of published studies. A brief discussion of the ways in which publication bias and professional incentives may distort the research literature in school psychology is contrasted against the importance of replications and preregistration of research (i.e., registered reports) as self-correcting mechanisms for research in school psychology. The limitations of current practices, coupled with the importance of registered reports and replications as self-correcting mechanisms, provide the context for this ongoing initiative in the Journal of School Psychology. Processes for manuscript submission, review, and publication are presented to encourage researchers to preregister studies and submit replications for publication.

psychology, educational

Update to the Journal of Water Resources Planning and Management's Reproducible Results Policy

Journal of Water Resources Planning and Management's Reproducibility Review Program: Accomplishments, Lessons, and Next Steps

Assessing data availability and research reproducibility in hydrology and water resources

National-Scale Optimized Design of Cost-Effective Water Supply and Transfer Systems

Repeatability, Reproducibility, Replicability, Reusability (4R) in Journals' Policies and Software/Data Management in Scientific Publications: A Survey, Discussion, and Perspectives

Partnering With Authors to Enhance Reproducibility at JASA

Erratum for "The Role of Reservoir Reoperation to Mitigate Climate Change Impacts on Hydropower and Environmental Water Demands"

The Basic Problem of Contemporary Water Resources Environment Research: Translation and Analysis from The English Practice Report on Water Resources

Reshaping the publication process: Addiction Research and Theory joins Peer Community In Registered Reports

Balancing Effectiveness and Equity in Sustainable Water Management Transitions: Case of the Miami-Dade Water and Sewer Department

Erratum for the Research Article: “The intensification of the water footprint of hydraulic fracturing” by A. J. Kondash, N. E. Lauer and A. Vengosh

Raising the bar for reproducible science at the U.S. Environmental Protection Agency Office of Research and Development

Influence of Subseasonal-to-Annual Water Supply Forecasts on Many-Objective Water System Robustness under Long-Term Change

The past, present and future of Registered Reports

Outstanding Reviewers for Environmental Science: Water Research & Technology in 2018

Watershed Science: Coupling hydrological science and water resources management

Optimized suspect screening approach for a comprehensive assessment of the impact of best management practices in reducing micropollutants transport in the Potomac River watershed

Reflecting on a dynamic biennium: The Annals of Regional Science 2022–2023

Sustainable computational science: the ReScience initiative

Registered Reports in the Journal of Sports Sciences

Registered reports and replications: An ongoing Journal of School Psychology initiative