Abstract:Abstract Objectives To synthesise research investigating data and code sharing in medicine and health to establish an accurate representation of the prevalence of sharing, how this frequency has changed over time, and what factors influence availability. Design Systematic review with meta-analysis of individual participant data. Data sources Ovid Medline, Ovid Embase, and the preprint servers medRxiv, bioRxiv, and MetaArXiv were searched from inception to 1 July 2021. Forward citation searches were also performed on 30 August 2022. Review methods Meta-research studies that investigated data or code sharing across a sample of scientific articles presenting original medical and health research were identified. Two authors screened records, assessed the risk of bias, and extracted summary data from study reports when individual participant data could not be retrieved. Key outcomes of interest were the prevalence of statements that declared that data or code were publicly or privately available (declared availability) and the success rates of retrieving these products (actual availability). The associations between data and code availability and several factors (eg, journal policy, type of data, trial design, and human participants) were also examined. A two stage approach to meta-analysis of individual participant data was performed, with proportions and risk ratios pooled with the Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis. Results The review included 105 meta-research studies examining 2 121 580 articles across 31 specialties. Eligible studies examined a median of 195 primary articles (interquartile range 113-475), with a median publication year of 2015 (interquartile range 2012-2018). Only eight studies (8%) were classified as having a low risk of bias. Meta-analyses showed a prevalence of declared and actual public data availability of 8% (95% confidence interval 5% to 11%) and 2% (1% to 3%), respectively, between 2016 and 2021. For public code sharing, both the prevalence of declared and actual availability were estimated to be <0.5% since 2016. Meta-regressions indicated that only declared public data sharing prevalence estimates have increased over time. Compliance with mandatory data sharing policies ranged from 0% to 100% across journals and varied by type of data. In contrast, success in privately obtaining data and code from authors historically ranged between 0% and 37% and 0% and 23%, respectively. Conclusions The review found that public code sharing was persistently low across medical research. Declarations of data sharing were also low, increasing over time, but did not always correspond to actual sharing of data. The effectiveness of mandatory data sharing policies varied substantially by journal and type of data, a finding that might be informative for policy makers when designing policies and allocating resources to audit compliance. Systematic review registration Open Science Framework doi: 10.17605/OSF.IO/7SX8U .

The PLOS ONE collection on machine learning in health and biomedicine: Towards open code and open data

Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness

Unlocking biomedical data sharing: A structured approach with digital twins and artificial intelligence (AI) for open health sciences

Beyond the hype of big data and artificial intelligence: building foundations for knowledge and wisdom

What is the Impact of Releasing Code with Publications? Statistics from the Machine Learning, Robotics, and Control Communities

Who Owns the Data? Open Data for Healthcare

A critical moment in machine learning in medicine: on reproducible and interpretable learning

Machine Learning in Health Care: Ethical Considerations Tied to Privacy, Interpretability, and Bias

The MIMIC Code Repository: enabling reproducibility in critical care research

Machine learning and AI research for Patient Benefit: 20 Critical Questions on Transparency, Replicability, Ethics and Effectiveness

Built to Last? Reproducibility and Reusability of Deep Learning Algorithms in Computational Pathology

Do no harm: a roadmap for responsible machine learning for health care

Prevalence and predictors of data and code sharing in the medical and health sciences: systematic review with meta-analysis of individual participant data

Diversity and inclusion: A hidden additional benefit of Open Data

Questionable practices in data and code sharing policy in high-profile medical journal and research

Open-Source Clinical Machine Learning Models: Critical Appraisal of Feasibility, Advantages, and Challenges

The impact of commercial health datasets on medical research and health-care algorithms

Privacy-preserving machine learning for healthcare: open challenges and future perspectives

Machine learning for integrating data in biology and medicine: Principles, practice, and opportunities

Copycats: the many lives of a publicly available medical imaging dataset

Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data