Transparency in epidemiological analyses of cohort data - A case study of the Norwegian Mother, Father, and Child cohort study (MoBa)
Timo B Roettger,Adrian Dahl Askelund,Viktoria B Birkenæs,Ludvig Daae Bjørndal,Agata Bochynska,Bernt Damian Glaser,Tamara Kalandadze,Max Korbmacher,Ivana Malovic,Julien Mayor,Pravesh Parekh,Daniel S Quintana,Laurie J. Hannigan
DOI: https://doi.org/10.1101/2024.12.05.24318481
2024-12-06
Abstract:Background: Epidemiological research is central to our understanding of health and disease. Secondary analysis of cohort data is an important tool in epidemiological research, but is vulnerable to practices that can reduce the validity and robustness of results. As such, adopting measures to increase the transparency and reproducibility of secondary data analysis is paramount to ensuring the robustness and usefulness of findings. The uptake of such practices has not yet been systematically assessed.
Methods: Using the Norwegian Mother, Father and Child Cohort study (MoBa; Magnus et al., 2006, 2016) as a case study, we assessed the prevalence of the following reproducible practices in publications between 2007 and 2023: preregistering secondary analyses, sharing of synthetic data, additional materials, and analysis scripts, conducting robustness checks, directly replicating previously published studies, declaring conflicts of interest and publishing publicly available versions of the paper.
Results: Preregistering secondary data analysis was only found in 0.4% of articles. No articles used synthetic data sets. Sharing practices of additional data (2.3%), additional materials (3.4%) and analysis scripts (4.2%) were rare. Several practices, including data and analysis sharing, preregistration and robustness checks became more frequent over time. Based on these assessments, we present a practical example for how researchers might improve transparency and reproducibility of their research.
Conclusions: The present assessment demonstrates that some reproducible practices are more common than others, with some practices being virtually absent. In line with a broader shift towards open science, we observed an increasing use of reproducible research practices in recent years. Nonetheless, the large amount of analytical flexibility offered by cohorts such as MoBa places additional responsibility on researchers to adopt such practices with urgency, to both ensure the robustness of their findings and earn the confidence of those using them. A particular focus in future efforts should be put on practices that help mitigating bias due to researcher degrees of freedom, namely, preregistration, transparent sharing of analysis scripts, and robustness checks. We demonstrate by example that challenges in implementing reproducible research practices in analysis of secondary cohort data, even including those associated with data sharing, can be meaningfully overcome.