Abstract:Non-parametric two-sample tests based on energy distance or maximum mean discrepancy are widely used statistical tests for comparing multivariate data from two populations. While these tests enjoy desirable statistical properties, their test statistics can be expensive to compute as they require the computation of 3 distinct Euclidean distance (or kernel) matrices between samples, where the time complexity of each of these computations (namely, $O(n_{x}^2 p)$, $O(n_{y}^2 p)$, and $O(n_{x} n_{y} p)$) scales quadratically with the number of samples ($n_x$, $n_y$) and linearly with the number of variables ($p$). Since the standard permutation test requires repeated re-computations of these expensive statistics it's application to large datasets can become unfeasible. While several statistical approaches have been proposed to mitigate this issue, they all sacrifice desirable statistical properties to decrease the computational cost (e.g., trade computation speed by a decrease in statistical power). A better computational strategy is to first pre-compute the Euclidean distance (kernel) matrix of the concatenated data, and then permute indexes and retrieve the corresponding elements to compute the re-sampled statistics. While this strategy can reduce the computation cost relative to the standard permutation test, it relies on the computation of a larger Euclidean distance (kernel) matrix with complexity $O((n_x + n_y)^2 p)$. In this paper, we present a novel computationally efficient permutation algorithm which only requires the pre-computation of the 3 smaller matrices and achieves large computational speedups without sacrificing finite-sample validity or statistical power. We illustrate its computational gains in a series of experiments and compare its statistical power to the current state-of-the-art approach for balancing computational cost and statistical performance.

Permutation Tests for Two-Sample Means of High-dimensional Data

A PERMUTATION TEST FOR TWO-SAMPLE MEANS AND SIGNAL IDENTIFICATION OF HIGH-DIMENSIONAL DATA

A New Method for Multi-Sample High-Dimensional Covariance Matrices Test Based on Permutation

Two-sample Test in High Dimensions Through Random Selection.

Multiple Permutation Test for High-Dimensional Data: a Components-Combined Algorithm

A Robust and Nonparametric Two-Sample Test in High Dimensions

Permutation Tests at Nonparametric Rates

Computationally efficient permutation tests for the multivariate two-sample problem based on energy distance or maximum mean discrepancy statistics

Simulation-Based Hypothesis Testing of High Dimensional Means Under Covariance Heterogeneity

Advances on Permutation Multivariate Analysis of Variance for big data

A P-Value Based Dimensionality Reduction Test for High Dimensional Means

Neyman's Truncation Test for Two-Sample Means under High Dimensional Setting

Two-Sample Test for Sparse High Dimensional Multinomial Distributions

A two-sample test for high-dimensional data with applications to gene-set testing

Two-sample high dimensional mean test based on prepivots

Nonparametric Two-Sample Tests of High Dimensional Mean Vectors Via Random Integration

Testing the Mean Vector for High-Dimensional Data

Two-Sample Tests for High Dimensional Means with Thresholding and Data Transformation

Permutation tests using arbitrary permutation distributions

Nonparametric High-Dimensional Multi-Sample Tests based on Graph Theory

Permutation tests for multivariate location problems