Abstract:Rapid advances in computing technology over the past few decades have spurred two extraordinary phenomena in science: large-scale and high-throughput data collection coupled with the creation and implementation of complex statistical algorithms for data analysis. Together, these two phenomena have brought about tremendous advances in scientific discovery but have also raised two serious concerns, one relatively new and one quite familiar. The complexity of modern data analyses raises questions about the reproducibility of the analyses, meaning the ability of independent analysts to re-create the results claimed by the original authors using the original data and analysis techniques. While seemingly a straightforward concept, reproducibility of analyses is typically thwarted by the lack of availability of the data and computer code that were used in the analyses. A much more general concern is the replicability of scientific findings, which concerns the frequency with which scientific claims are confirmed by completely independent investigations. While the concepts of reproduciblity and replicability are related, it is worth noting that they are focused on quite different goals and address different aspects of scientific progress. In this review, we will discuss the origins of reproducible research, characterize the current status of reproduciblity in public health research, and connect reproduciblity to current concerns about replicability of scientific findings. Finally, we describe a path forward for improving both the reproducibility and replicability of public health research in the future.

Reasons, Challenges and Some Tools for Doing Reproducible Research in Transportation Research

Toward Reusable Science with Readable Code and Reproducibility

Why is Your Paper Rejected? Lessons Learned from over 5000 Rejected Transportation Papers

Reproducibility in Research: Systems, Infrastructure, Culture

"Can I Implement Your Algorithm?": A Model for Reproducible Research Software

Five creative ways to promote reproducible science

Building a Culture of Reproducibility in Academic Research

Provide Proactive Reproducible Analysis Transparency with Every Publication

Primer on Reproducible Research in R: Enhancing Transparency and Scientific Rigor

A Collaborative Approach to Computational Reproducibility

Open Reproducible Publication Research

Packaging code and data for reproducible research: A case study of journey time statistics

Promoting computationally reproducible research in applied linguistics: Recommended practices and considerations

Reproducible research practices: A tool for effective and efficient leadership in collaborative statistics

A large-scale study on research code quality and execution

ReproduceMe: Lessons from a pilot project on computational reproducibility

Tools and Recommendations for Reproducible Teaching

Reproducible Research: A Retrospective

Laying foundations to quantify the "Effort of Reproducibility"

Replicable and extensible spatial data acquisition

Administrative social science data: The challenge of reproducible research