Abstract:In this work, we investigate the importance of explicitly accounting for cross-trial variability in neuroimaging data analysis. To attempt to obtain reliable estimates in a task-based experiment, each condition is usually repeated across many trials. The investigator may be interested in (a) condition-level effects, (b) trial-level effects, or (c) the association of trial-level effects with the corresponding behavior data. The typical strategy for condition-level modeling is to create one regressor per condition at the subject level with the underlying assumption that responses do not change across trials. In this methodology of complete pooling, all cross-trial variability is ignored and dismissed as random noise that is swept under the rug of model residuals. Unfortunately, this framework invalidates the generalizability from the confine of specific trials (e.g., particular faces) to the associated stimulus category ("face"), and may inflate the statistical evidence when the trial sample size is not large enough. Here we propose an adaptive and computationally tractable framework that meshes well with the current two-level pipeline and explicitly accounts for trial-by-trial variability. The trial-level effects are first estimated per subject through no pooling. To allow generalizing beyond the particular stimulus set employed, the cross-trial variability is modeled at the population level through partial pooling in a multilevel model, which permits accurate effect estimation and characterization. Alternatively, trial-level estimates can be used to investigate, for example, brain-behavior associations or correlations between brain regions. Furthermore, our approach allows appropriate accounting for serial correlation, handling outliers, adapting to data skew, and capturing nonlinear brain-behavior relationships. By applying a Bayesian multilevel model framework at the level of regions of interest to an experimental dataset, we show how multiple testing can be addressed and full results reported without arbitrary dichotomization. Our approach revealed important differences compared to the conventional method at the condition level, including how the latter can distort effect magnitude and precision. Notably, in some cases our approach led to increased statistical sensitivity. In summary, our proposed framework provides an effective strategy to capture trial-by-trial responses that should be of interest to a wide community of experimentalists.

Towards Efficient FMRI Data Re-Use: Can We Run Between-Group Analyses with Datasets Processed Differently with SPM?

On the validity of fMRI studies with subject-level data processed through different pipelines

PAIR Comparison Between Two Within-Group Conditions of Resting-State Fmri Improves Classification Accuracy

Analysis of BOLD fMRI Signal Preprocessing Pipeline on Different Datasets while Reducing False Positive Rates

Moving Beyond Processing and Analysis-Related Variation in Neuroscience

The HCP multi-pipeline dataset: an opportunity to investigate analytical variability in fMRI data analysis

An Empirical Comparison of SPM Preprocessing Parameters to the Analysis of fMRI Data

Small effect size leads to reproducibility failure in resting-state fMRI studies

A sensitivity analysis to quantify the impact of neuroimaging preprocessing strategies on subsequent statistical analyses

Modelling the data and not the images in FMRI

Selection of a Model of Cerebral Activity for fMRI Group Data Analysis

FMRI processing with AFNI: Some comments and corrections on “Exploring the Impact of Analysis Software on Task fMRI Results”

To pool or not to pool: Can we ignore cross-trial variability in FMRI?

Impact of analytic decisions on test-retest reliability of individual and group estimates in functional magnetic resonance imaging: a multiverse analysis using the monetary incentive delay task

FMRI Clustering in AFNI: False-Positive Rates Redux

Applications of multivariate modeling to neuroimaging group analysis: a comprehensive alternative to univariate general linear model

Uncovering communities of pipelines in the task-fMRI analytical space

Choice of Voxel-based Morphometry Processing Pipeline Drives Variability in the Location of Neuroanatomical Brain Markers

Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates

Statistical power and prediction accuracy in multisite resting-state fMRI connectivity

Statistical significance in DTI group analyses: How the choice of the estimator can inflate effect sizes