Abstract:With the recognized crisis of credibility in scientific research, there is a growth of reproducibility studies in computer science, and although existing surveys have reviewed reproducibility from various perspectives, especially very specific technological issues, they do not address the author-publisher relationship in the publication of reproducible computational scientific articles. This aspect requires significant attention because it is the basis for reliable research. We have found a large gap between the reproducibility-oriented practices, journal policies, recommendations, publisher artifact Description/Evaluation guidelines, submission guides, technological reproducibility evolution, and its effective adoption to contribute to tackling the crisis. We conducted a narrative survey, a comprehensive overview and discussion identifying the mutual efforts required from Authors, Journals, and Technological actors to achieve reproducibility research. The relationship between authors and scientific journals in their mutual efforts to jointly improve the reproducibility of scientific results is analyzed. Eventually, we propose recommendations for the journal policies, as well as a unified and standardized Reproducibility Guide for the submission of scientific articles for authors. The main objective of this work is to analyze the implementation and experiences of reproducibility policies, techniques and technologies, standards, methodologies, software, and data management tools required for scientific reproducible publications. Also, the benefits and drawbacks of such an adoption, as well as open challenges and promising trends, to propose possible strategies and efforts to mitigate the identified gaps. To this purpose, we analyzed 200 scientific articles, surveyed 16 Computer Science journals, and systematically classified them according to reproducibility strategies, technologies, policies, code citation, and editorial business. We conclude there is still a reproducibility gap in scientific publications, although at the same time also the opportunity to reduce this gap with the joint effort of authors, publishers, and technological providers.

Reproducibility Needs Reshape Scientific Data Governance

Provide Proactive Reproducible Analysis Transparency with Every Publication

Administrative social science data: The challenge of reproducible research

Designing for Reproducibility: A Qualitative Study of Challenges and Opportunities in High Energy Physics

Repeatability, Reproducibility, Replicability, Reusability (4R) in Journals' Policies and Software/Data Management in Scientific Publications: A Survey, Discussion, and Perspectives

Reproducibility, Replicability, and Transparency in Research: What 430 Professors Think in Universities across the USA and India

Reproducibility in Research: Systems, Infrastructure, Culture

Data management to support reproducible research

Raising the bar for reproducible science at the U.S. Environmental Protection Agency Office of Research and Development

Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition

Reproducible Research: A Retrospective

A Decision Tree to Shepherd Scientists through Data Retrievability

A Collaborative Approach to Computational Reproducibility

Reproducible Research Can Still Be Wrong: Adopting a Prevention Approach

Five creative ways to promote reproducible science

Recommendations to enhance rigor and reproducibility in biomedical research

Eleven strategies for making reproducible research and open science training the norm at research institutions

Reproducibility in the Classroom

Biomedical Data Repository Concepts and Management Principles

Promoting Open Science Through Research Data Management

Guidelines and Initiatives for Good Research Practice