Abstract:One of the key ingredients of scientific progress is the ability to repeat, replicate and reproduce independently important scientific findings. Recently, independent groups failed to replicate the results of several experiments in various research areas, opening the so-called "reproducibility crisis". The reasons behind these failures may be motivated by the excessive trust given to the results obtained by digital computers. Indeed, little attention was given to the implementation of a principal algorithm, and method or to the variation introduced by the use of different software, and hardware systems or to how difficult a finding can be recover after weeks or years or to the precision level one had performed a computational experiment (Donoho et al., 2009;Peng, 2011).To extricate this tight tangled set of terms, it is important to precisely define the meaning of reproducing, replicating, and repeating with the terminology long established in experimental sciences (Plesser, 2018). The Association for Computing Machinery (ACM) has adopted the following definitions for the three highly used terms on research (Association for Computing Machinery, 2016):Repeatability (Same team, same experimental setup, same data): Τhe measurements (findings) can be obtained with precision by the same team, using the same measurement procedure (experimental protocol), the same measuring system, under the same operating conditions (e.g. neuroimaging system like MRI 3T with the same set-up, time of the day, etc.), in the same location (Lab) following a multiple trial acquisition protocol. For solely computational experiments, it practically means that a researcher can reliably repeat his/her own computations.Replicability (Different team, same experimental setup, different data): Τhe measurements (findings) can be obtained with precision by a different team, using the same measurement procedure (experimental protocol), the same measuring system, under the same operating conditions (e.g. neuroimaging system like MRI 3T with the same set-up, time of the day, etc.), in the same location (Lab) following a multiple trial acquisition protocol. For solely computational experiments, it practically means that an independent group can obtain the same result by employing the author's own experimental artifacts.Reproducibility (Different team, different experimental setup, different data): Τhe measurements (findings) can be obtained with precision by a different team, using a different measurement procedure (experimental protocol), a different measuring system, under similar operating conditions (e.g. neuroimaging system like MRI 3T with the same set-up, time of the day, etc.) in a different location (Lab) following a multiple trial acquisition protocol. For solely computational experiments, it practically means that an independent group can obtain the same result using experimental artifacts produced completely independently from the author's artifacts.For more information, an interested researcher can read informative studies discussing this terminology (Crook et al., 2013;Goodman et al., 2016;Nichols et al., 2017).In computational neuroscience, there are two types of studies: simulation experiments and advanced analyses of experimental data. In both types of studies, methods reproducibility refers to obtaining the same results when running the same code again. However, this type of reproducibility demands access to experimental data, code, and simulation specifications (Botvinik-Nezer and Wager, 2023). Results reproducibility demands access to the experimental data, but the analysis can be realized by using different pipelines (combination of methods, code) e.g. analysis packages or neural simulators.In the methodological part of a study, data analysis demands a lot of decisions. More recently, 70 independent analysis teams tested 9 prespecified hypotheses using the same taskfunctional magnetic resonance imaging (fMRI) dataset (Botvinik-Nezer et al., 2020). The 70 teams selected 70 different analytical pipelines, and this variation affected the results, including the statistical maps and conclusions drawn regarding the preselected hypotheses tested. In a recent study, Luppi et al. (2024) systematically evaluated 768 data-processing pipelines for network construction from resting-state functional MRI, evaluating the effect of brain parcellation, global signal regression, and connectivity definition.Several organizations worldwide have tried to increase awareness about the importance of reproducibility and replicability in different disciplines in recent years (e.g. www.repro4everyone.org, Global Reproducibility Networks), including open-source repositories for research resources (e.g. Zenodo), protocols, source code (e.g. GitHub), datasets, etc. Fig. 1 illustrates schematically the key ingredients to make a research study reproducible and replicable (Auer et al., 2021).In recent years, myr -Abstract Truncated-

Reproducibility Made Easy: A Tool for Methodological Transparency and Efficient Standardized Reporting based on the proposed MRSinMRS Consensus

Nontargeted Analysis Study Reporting Tool: A Framework to Improve Research Transparency and Reproducibility

Reproducible manuscript preparation with RMarkdown application to JMSACL and other Elsevier Journals

The Reporting Recommendations Intended for Pharmaceutical Risk Minimization Evaluation Studies: Standards for Reporting of Implementation Studies Extension (RIMES-SE)

Strengthening the reporting of observational studies in epidemiology using mendelian randomisation (STROBE-MR): explanation and elaboration

An Open-Source R Package for Detection of Adverse Events Under-Reporting in Clinical Trials: Implementation and Validation by the IMPALA (Inter coMPany quALity Analytics) Consortium

ReproduceMe: Lessons from a pilot project on computational reproducibility

Assessing Study Reproducibility through M 2 RI : A Novel Approach for Large-scale High-throughput Association Studies

ReProMSig: an Integrative Platform for Development and Application of Reproducible Multivariable Models for Cancer Prognosis Supporting Guideline-Based Transparent Reporting.

Toward Reusable Science with Readable Code and Reproducibility

Data-sharing and re-analysis for main studies assessed by the European Medicines Agency—a cross-sectional study on European Public Assessment Reports

Raising the value of research studies in psychological science by increasing the credibility of research reports: the transparent Psi project

Creating reproducible dynamic reports using data from the Secure Anonymised Information Linkage (SAIL) Databank

RRegrs: an R package for computer-aided model selection with multiple regression models

Editorial: Reproducible analysis in neuroscience

Data management to support reproducible research

The MIMIC Code Repository: enabling reproducibility in critical care research

ENCORE: a practical implementation to improve reproducibility and transparency of computational research

ACCORD (ACcurate COnsensus Reporting Document): A reporting guideline for consensus methods in biomedicine developed via a modified Delphi

A Collaborative Approach to Computational Reproducibility

Reshaping the publication process: Addiction Research and Theory joins Peer Community In Registered Reports