Abstract:Network neuroscience has emerged as an indispensable tool for studying brain structure and function. Currently, the network-based statistic (NBS) procedure is widely used for dealing with massive multiple testing/comparison problems in brain networks. However, the NBS requires choosing a hard cluster-forming threshold, lacking objective rules. A powerful and flexible statistical framework is urgently needed with growing interest in finer-grained network explorations across modalities and scales. Here, we introduce a permutation-based framework—”Threshold-Free Network-Oriented Statistics” (TFNOS). It integrates two “threshold-free” pathways: traversing all cluster-forming thresholds (TT) and using predefined clusters (PC). The TT procedure, building upon the threshold-free network-based statistics, requires setting additional parameters. The PC procedures comprise six variants given the degree of freedom in pooling data, null distribution construction, and controlled error rate. Using numerical simulations, we evaluated the performance of the TT procedure under 600 parameter combinations, then benchmarked TFNOS procedures and baselines across different topologies of effects, sample sizes, and effect sizes, and finally provided illustrative examples with real data. We offer recommended parameter values that allow the TT procedure to stably maintain leading power, while empirically controlling the false discovery rate (FDR) beyond only weakly controlling the familywise error rate (FWER). Notably, the relevant parameters commonly employed in the field appear overly liberal. Furthermore, for the PC procedures, FDR-controlling variants showed improved power compared to FWER-controlling variants, and some of them are simple but do not compromise power. The nonparametric PC procedures allow the selection of any test statistics considered appropriate. Overall, the TFNOS is a generalized framework for inference on edges/nodes of undirected/directed brain networks. We provide empirical and principled criteria for selecting appropriate procedures and may enhance the reproducibility and sensitivity of future brain research.

Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate

Adaptive Thresholding of Statistical Parametric Map in Fmri

Seeing more than the Tip of the Iceberg: Approaches to Subthreshold Effects in Functional Magnetic Resonance Imaging of the Brain

DeepFDR: A Deep Learning-based False Discovery Rate Control Method for Neuroimaging Data

Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates

Threshold-Free Network-Oriented Statistics in Neuroscience

On the logic of hypothesis testing in functional imaging

Modified test statistics by inter-voxel variance shrinkage with an application to f MRI.

Proportional thresholding in resting-state fMRI functional connectivity networks and consequences for patient-control connectome studies: Issues and recommendations

The Problem of Thresholding in Small-World Network Analysis

Improving the precision of fMRI BOLD signal deconvolution with implications for connectivity analysis

Assessment and Optimization of Functional MRI Analyses

False discovery rate analysis of brain diffusion direction maps

A Computationally Efficient, Exploratory Approach to Brain Connectivity Incorporating False Discovery Rate Control, a Priori Knowledge, and Group Inference.

FMRI Clustering in AFNI: False-Positive Rates Redux

Multiple Comparison Procedures for Neuroimaging Genomewide Association Studies

Bayesian Inference for Brain Activity from Functional Magnetic Resonance Imaging Collected at Two Spatial Resolutions

Controlling false discovery rate for mediator selection in high-dimensional data

Is Bonferroni correction more sensitive than Random Field Theory for most fMRI studies?

Comparison of data-driven thresholding methods using directed functional brain networks

Multiscale statistical testing for connectome-wide association studies in fMRI