Abstract:Requiring only minimal assumptions for validity, nonparametric permutation testing provides a flexible and intuitive methodology for the statistical analysis of data from functional neuroimaging experiments, at some computational expense. Introduced into the functional neuroimaging literature by Holmes et al. ([1996]: J Cereb Blood Flow Metab 16:7-22), the permutation approach readily accounts for the multiple comparisons problem implicit in the standard voxel-by-voxel hypothesis testing framework. When the appropriate assumptions hold, the nonparametric permutation approach gives results similar to those obtained from a comparable Statistical Parametric Mapping approach using a general linear model with multiple comparisons corrections derived from random field theory. For analyses with low degrees of freedom, such as single subject PET/SPECT experiments or multi-subject PET/SPECT or fMRI designs assessed for population effects, the nonparametric approach employing a locally pooled (smoothed) variance estimate can outperform the comparable Statistical Parametric Mapping approach. Thus, these nonparametric techniques can be used to verify the validity of less computationally expensive parametric approaches. Although the theory and relative advantages of permutation approaches have been discussed by various authors, there has been no accessible explication of the method, and no freely distributed software implementing it. Consequently, there have been few practical applications of the technique. This article, and the accompanying MATLAB software, attempts to address these issues. The standard nonparametric randomization and permutation testing ideas are developed at an accessible level, using practical examples from functional neuroimaging, and the extensions for multiple comparisons described. Three worked examples from PET and fMRI are presented, with discussion, and comparisons with standard parametric approaches made where appropriate. Practical considerations are given throughout, and relevant statistical concepts are expounded in appendices.

Permute-match tests: Detecting significant correlations between time series despite nonstationarity and limited replicates

Testing for Parallelism among Trends in Multiple Time Series.

A rigorous and versatile statistical test for correlations between stationary time series

Subtle methodological variations substantially impact correlation test results in ecological time series

A PERMUTATION TEST FOR TWO-SAMPLE MEANS AND SIGNAL IDENTIFICATION OF HIGH-DIMENSIONAL DATA

A novel method to accurately calculate statistical significance of local similarity analysis for high-throughput time series.

Robust non-parametric tests for complex-repeated measures problems in ophthalmology

Least Squares-Based Permutation Tests in Time Series

Fast Approximation of Small P-values in Permutation Tests by Partitioning the Permutations

A robust Spearman correlation coefficient permutation test

JUMP: replicability analysis of high-throughput experiments with applications to spatial transcriptomic studies

Sequential Permutation Testing of Random Forest Variable Importance Measures

Randomized p-values for multiple testing and their application in replicability analysis

Enhanced adaptive permutation test with negative binomial distribution in genome-wide omics datasets

Statistical Analysis of Data Repeatability Measures

Nonparametric permutation tests for functional neuroimaging: A primer with examples

Efficient Approximation of Statistical Significance in Local Trend Analysis of Dependent Time Series

Efficiently estimating small p-values in permutation tests using importance sampling and cross-entropy method

Kernel-Based Partial Permutation Test for Detecting Heterogeneous Functional Relationship

The Classification Permutation Test: A Nonparametric Test for Equality of Multivariate Distributions

Detecting Multiple Replicating Signals using Adaptive Filtering Procedures