Abstract:The problem of robust hypothesis testing is studied, where under the null and the alternative hypotheses, the data-generating distributions are assumed to be in some uncertainty sets, and the goal is to design a test that performs well under the worst-case distributions over the uncertainty sets. In this paper, uncertainty sets are constructed in a data-driven manner using kernel method, i.e., they are centered around empirical distributions of training samples from the null and alternative hypotheses, respectively; and are constrained via the distance between kernel mean embeddings of distributions in the reproducing kernel Hilbert space, i.e., maximum mean discrepancy (MMD). The Bayesian setting and the Neyman-Pearson setting are investigated. For the Bayesian setting where the goal is to minimize the worst-case error probability, an optimal test is firstly obtained when the alphabet is finite. When the alphabet is infinite, a tractable approximation is proposed to quantify the worst-case average error probability, and a kernel smoothing method is further applied to design test that generalizes to unseen samples. A direct robust kernel test is also proposed and proved to be exponentially consistent. For the Neyman-Pearson setting, where the goal is to minimize the worst-case probability of miss detection subject to a constraint on the worst-case probability of false alarm, an efficient robust kernel test is proposed and is shown to be asymptotically optimal. Numerical results are provided to demonstrate the performance of the proposed robust tests.

A More Robust t-Test

Efficient three-stage $t$-tests

A robust alternative to the Lilliefors test of normality

Powerful t-tests in the presence of nonclassical measurement error

A NEW PERSPECTIVE ON ROBUST M-ESTIMATION: FINITE SAMPLE THEORY AND APPLICATIONS TO DEPENDENCE-ADJUSTED MULTIPLE TESTING

The New and Improved Two-Sample t Test

Finite sample t-tests for high-dimensional means

Robustness of the T Test: A Guide for Researchers on Effect of Violations of Assumptions

Robustness and accuracy of methods for high dimensional data analysis based on Student's t statistic

An Extreme-Value Approach for Testing the Equality of Large U-Statistic Based Correlation Matrices

On the feasibility of parsimonious variable selection for Hotelling's $T^2$-test

A noniterative sample size procedure for tests based on t distributions

Model-averaged Bayesian t tests

Enhanced power enhancements for testing many moment equalities: Beyond the $2$- and $\infty$-norm

t-tests, non-parametric tests, and large studies—a paradox of statistical practice?

Simultaneous critical values for $t$-tests in very high dimensions

A robust test approach for equality of mean vectors of two independent groups under the multivariate Behrens-Fisher problem

The effects of violations of assumptions underlying the t test.

A Kernel Method for the Two-Sample Problem

Small sample sizes: A big data problem in high-dimensional data analysis

Kernel Robust Hypothesis Testing