Bias in data-driven estimates of the reproducibility of univariate brain-wide association studies.

Charles David Gutierrez Burns,Alessio Fracasso,Guillaume A Rousselet

DOI: https://doi.org/10.1101/2023.09.21.558661

2024-12-10

Abstract:Recent studies have used big neuroimaging datasets to answer an important question: how many subjects are required for reproducible brain-wide association studies? These data- driven approaches could be considered a framework for testing the reproducibility of several neuroimaging models and measures. Here we test part of this framework, namely estimates of statistical errors of univariate brain-behaviour associations obtained from resampling large datasets with replacement. We demonstrate that reported estimates of statistical errors are largely a consequence of bias introduced by random effects when sampling with replacement close to the full sample size. We show that future meta-analyses can largely avoid these biases by only resampling up to 10% of the full sample size. We discuss implications that reproducing mass-univariate association studies requires tens-of- thousands of participants, urging researchers to adopt other methodological approaches.

Neuroscience