Abstract:Non-parametric two-sample tests based on energy distance or maximum mean discrepancy are widely used statistical tests for comparing multivariate data from two populations. While these tests enjoy desirable statistical properties, their test statistics can be expensive to compute as they require the computation of 3 distinct Euclidean distance (or kernel) matrices between samples, where the time complexity of each of these computations (namely, $O(n_{x}^2 p)$, $O(n_{y}^2 p)$, and $O(n_{x} n_{y} p)$) scales quadratically with the number of samples ($n_x$, $n_y$) and linearly with the number of variables ($p$). Since the standard permutation test requires repeated re-computations of these expensive statistics it's application to large datasets can become unfeasible. While several statistical approaches have been proposed to mitigate this issue, they all sacrifice desirable statistical properties to decrease the computational cost (e.g., trade computation speed by a decrease in statistical power). A better computational strategy is to first pre-compute the Euclidean distance (kernel) matrix of the concatenated data, and then permute indexes and retrieve the corresponding elements to compute the re-sampled statistics. While this strategy can reduce the computation cost relative to the standard permutation test, it relies on the computation of a larger Euclidean distance (kernel) matrix with complexity $O((n_x + n_y)^2 p)$. In this paper, we present a novel computationally efficient permutation algorithm which only requires the pre-computation of the 3 smaller matrices and achieves large computational speedups without sacrificing finite-sample validity or statistical power. We illustrate its computational gains in a series of experiments and compare its statistical power to the current state-of-the-art approach for balancing computational cost and statistical performance.

A PERMUTATION TEST FOR TWO-SAMPLE MEANS AND SIGNAL IDENTIFICATION OF HIGH-DIMENSIONAL DATA

Fast Approximation of Small P-values in Permutation Tests by Partitioning the Permutations

Generalized Permutation Framework for Testing Model Variable Significance

Computationally efficient permutation tests for the multivariate two-sample problem based on energy distance or maximum mean discrepancy statistics

Fast permutation tests and related methods, for association between rare variants and binary outcomes

Efficiently estimating small p-values in permutation tests using importance sampling and cross-entropy method

Two-Sample Tests for High Dimensional Means with Thresholding and Data Transformation

Enhanced adaptive permutation test with negative binomial distribution in genome-wide omics datasets

A two-sample test for high-dimensional data with applications to gene-set testing

Accurate and Fast Small P-Value Estimation for Permutation Tests in High-Throughput Genomic Data Analysis with the Cross-Entropy Method.

Permutation Tests at Nonparametric Rates

The Classification Permutation Test: A Nonparametric Test for Equality of Multivariate Distributions

Kernel-smoothed permutation for extreme P-value estimation in genetic association studies

Advances on Permutation Multivariate Analysis of Variance for big data

Simulation-Based Hypothesis Testing of High Dimensional Means Under Covariance Heterogeneity

Two-sample high dimensional mean test based on prepivots

Permutation tests for the equality of covariance operators of functional data with applications to evolutionary biology

Two-Sample Tests for High-Dimensional Linear Regression with an Application to Detecting Interactions.

Bayesian Optimal Two-sample Tests in High-dimension

Permute-match tests: Detecting significant correlations between time series despite nonstationarity and limited replicates

Residual Permutation Test for High-Dimensional Regression Coefficient Testing