Abstract:Population neuroscience is a discipline generated by the intersection of neuroscience (brain), genomics (gene), and epidemiology (environment), providing new perspectives and methods for studying a full picture of human brain structure and function. Resting state functional magnetic resonance imaging (rsfMRI) has become one of the most widely used tool for collecting brain phenotype information in population neuroscience. Its measurement of spontaneous neural activity in the human brain is safe, reliable, and easy to operate and implement in large-scale studies. RsfMRI can reveal the patterns of cognitive activity and psychiatric conditions. Its reliability is a fundamental requirement for measuring individual differences in the spontaneous brain activity. However, the importance of reliability in rsfMRI has not been fully recognized by the field. This article first provides a brief introduction to the definition and calculation of reliability of rsfMRI. By reviewing the literature, we describe the research progresses on the reliability of rsfMRI. Intraclass correlation (ICC) is the most ideal indicator for measuring the reliability of rsfMRI and used in most current research on rsfMRI to calculate the test-retest reliability of rsfMRI measurements. The overall proportion of reliability research on rsfMRI is small, and empirical research on reliability needs to be strengthened. The attention to the reliability of rsfMRI in the field is constantly increasing, and the reliability of rsfMRI is often involved and discussed. The overall measurement reliability level of rsfMRI is mostly below 0.6, mostly around 0.4; The reliability obtained from different studies and measurement indicators varies greatly, and there is also significant heterogeneity in the reliability between brain networks. Secondly, a study on the reliability of rsfMRI was proposed from three dimensions: demographic characteristics, sample size, and measurement arrangement. The impact of various factors in these three dimensions on the reliability of rsfMRI was summarized, including gender, age, sample size, scanner, magnetic field intensity, scanning parameters, physiological noise, head movement, body cognitive changes, open/closed eye status, retest interval, and scanning duration factors such as data preprocessing and functional indicators have varying degrees of impact on the reliability of rsfMRI. Finally, a guideline for the reliability study of population neuroscience with rsfMRI is proposed: (1) Referring to the brain chart of the entire human life cycle and using standardized modeling based on population neuroscience; (2) When conducting rsfMRI, report reliability as a necessary content and provide measurement indicator reliability information; (3) Drawing on a large amount of open access rsfMRI data resources for research, increasing the sample size, avoiding subjects from closing their eyes and sleeping during scanning, and the scanning interval should not be too long, the shorter the better. It is necessary to extend the scanning time each time; (4) Continuously optimizing scanning parameters, reducing scanning noise, and conducting standardized scans to improve the comfort level of participants during scanning, in order to reduce physical movement and cognitive changes, and monitor the scanning process in real time to pay attention to individual physical and mental changes; (5) Adopting standardized procedures to standardize the data rest state function and preprocessing process, selecting calculation indicators and methods to achieve the goal of improving research reliability. To better serve the in-depth development of population neuroscience and the widespread use of rsfMRI, it is imperative to comprehensively reveal the reliability and influencing factors of rsfMRI. Population neuroscience still needs to improve research on the impact of demographic characteristics, sample size, and testing arrangements on the reliability of rsfMRI measurements. Soon, population neuroscience should also refer to disciplines such as psychometric to establish a complete set of rsfMRI data collection standards and guidelines, truly achieving stable and reliable measurement of data.

Small effect size leads to reproducibility failure in resting-state fMRI studies

Resting-state Fmri and Population Neuroscience: Progresses and Guidelines for Reliability Research

Reproducibility and Sensitivity of Resting-State Fmri in Patients with Parkinson's Disease Using Cross Validation-Based Data Censoring

Twenty-Five Cohorts of Major Depressive Disorder in China: Evidence for Reduced But Not Enhanced Default Mode Network Functional Connectivity

Reproducibility of Functional Brain Alterations in Major Depressive Disorder: Evidence from a Multisite Resting-State Functional MRI Study with 1,434 Individuals

The Effect of Low-Frequency Physiological Correction on the Reproducibility and Specificity of Resting-State fMRI Metrics: Functional Connectivity, ALFF, and ReHo

Inconsistency in Abnormal Brain Activity Across Cohorts of ADHD-200 in Children with Attention Deficit Hyperactivity Disorder

Effect Sizes of Associations Between Neuroimaging Measures and Affective Symptoms: A Meta‐analysis

Altered Resting-State Dynamic Functional Brain Networks in Major Depressive Disorder: Findings from the REST-meta-MDD Consortium

MRI economics: Balancing sample size and scan duration in brain wide association studies

Assessing Overall Reproducibility for Large-scale High-throughput MRI-based Association Studies

Scanning the horizon: towards transparent and reproducible neuroimaging research

Influence of sample size and analytic approach on stability and interpretation of brain‐behavior correlations in task‐related fMRI data

An illustration of reproducibility in neuroscience research in the absence of selective reporting

An Empirical Comparison of Meta- and Mega-Analysis with Data from the ENIGMA Obsessive-Compulsive Disorder Working Group.

Resting-state functional reorganization in Parkinson's disease: An activation likelihood estimation meta-analysis

Reproducibility in the Absence of Selective Reporting: an Illustration from Large-Scale Brain Asymmetry Research

Study design features increase replicability in brain-wide association studies

Standardizing the intrinsic brain: Towards robust measurement of inter-individual variation in 1000 functional connectomes.

Exploring the reproducibility of functional connectivity alterations in Parkinson's Disease

Grab‐AD: Generalizability and Reproducibility of Altered Brain Activity and Diagnostic Classification in Alzheimer's Disease