Abstract:Requiring only minimal assumptions for validity, nonparametric permutation testing provides a flexible and intuitive methodology for the statistical analysis of data from functional neuroimaging experiments, at some computational expense. Introduced into the functional neuroimaging literature by Holmes et al. ([1996]: J Cereb Blood Flow Metab 16:7-22), the permutation approach readily accounts for the multiple comparisons problem implicit in the standard voxel-by-voxel hypothesis testing framework. When the appropriate assumptions hold, the nonparametric permutation approach gives results similar to those obtained from a comparable Statistical Parametric Mapping approach using a general linear model with multiple comparisons corrections derived from random field theory. For analyses with low degrees of freedom, such as single subject PET/SPECT experiments or multi-subject PET/SPECT or fMRI designs assessed for population effects, the nonparametric approach employing a locally pooled (smoothed) variance estimate can outperform the comparable Statistical Parametric Mapping approach. Thus, these nonparametric techniques can be used to verify the validity of less computationally expensive parametric approaches. Although the theory and relative advantages of permutation approaches have been discussed by various authors, there has been no accessible explication of the method, and no freely distributed software implementing it. Consequently, there have been few practical applications of the technique. This article, and the accompanying MATLAB software, attempts to address these issues. The standard nonparametric randomization and permutation testing ideas are developed at an accessible level, using practical examples from functional neuroimaging, and the extensions for multiple comparisons described. Three worked examples from PET and fMRI are presented, with discussion, and comparisons with standard parametric approaches made where appropriate. Practical considerations are given throughout, and relevant statistical concepts are expounded in appendices.

Permutation Tests for Regression, ANOVA, and Comparison of Signals: The permuco Package

PERMUTOOLS: A MATLAB Package for Multivariate Permutation Testing

An Empirical Comparison of Parametric and Permutation Tests for Regression Analysis of Randomized Experiments

Robust non-parametric tests for complex-repeated measures problems in ophthalmology

Nonparametric permutation tests for functional neuroimaging: A primer with examples

Permutation Tests for Assessing Potential Non-Linear Associations between Treatment Use and Multivariate Clinical Outcomes

A note on extending Scheffé's modified multiple-comparison procedure to other analysis situations

New methods for multiple testing in permutation inference for the general linear model

The Classification Permutation Test: A Nonparametric Test for Equality of Multivariate Distributions

A PERMUTATION TEST FOR TWO-SAMPLE MEANS AND SIGNAL IDENTIFICATION OF HIGH-DIMENSIONAL DATA

Sequential Permutation Testing of Random Forest Variable Importance Measures

Multivariate quantile-based permutation tests with application to functional data

Permutation-based multiple testing when fitting many generalized linear models

Permutation tests for detecting and estimating mixtures in task performance within groups.

Permutation-based tests for discontinuities in event studies

Generalized Permutation Framework for Testing Model Variable Significance

Kernel-Based Partial Permutation Test for Detecting Heterogeneous Functional Relationship

Permutation tests for the equality of covariance operators of functional data with applications to evolutionary biology

The Exchangeability Assumption for Permutation Tests of Multiple Regression Models: Implications for Statistics and Data Science

Deriving Difference Between the Bayesian Networks Based Patterns of the Effective Connectivity Using Permutation Test in Fmri Studies

Permutation inference for canonical correlation analysis