Abstract:Machine learning models are known to be susceptible to adversarial attacks which can cause misclassification by introducing small but well designed perturbations. In this paper, we consider a classical hypothesis testing problem in order to develop fundamental insight into defending against such adversarial perturbations. We interpret an adversarial perturbation as a nuisance parameter, and propose a defense based on applying the generalized likelihood ratio test (GLRT) to the resulting composite hypothesis testing problem, jointly estimating the class of interest and the adversarial perturbation. While the GLRT approach is applicable to general multi-class hypothesis testing, we first evaluate it for binary hypothesis testing in white Gaussian noise under $\ell_{\infty}$ norm-bounded adversarial perturbations, for which a known minimax defense optimizing for the worst-case attack provides a benchmark. We derive the worst-case attack for the GLRT defense, and show that its asymptotic performance (as the dimension of the data increases) approaches that of the minimax defense. For non-asymptotic regimes, we show via simulations that the GLRT defense is competitive with the minimax approach under the worst-case attack, while yielding a better robustness-accuracy tradeoff under weaker attacks. We also illustrate the GLRT approach for a multi-class hypothesis testing problem, for which a minimax strategy is not known, evaluating its performance under both noise-agnostic and noise-aware adversarial settings, by providing a method to find optimal noise-aware attacks, and heuristics to find noise-agnostic attacks that are close to optimal in the high SNR regime.

Robust composite binary hypothesis testing via measure-transformed quasi score test

Measure Transformed Quasi Score Test with Application to Location Mismatch Detection

Measure-Transformed Quasi Maximum Likelihood Estimation

A NEW PERSPECTIVE ON ROBUST M-ESTIMATION: FINITE SAMPLE THEORY AND APPLICATIONS TO DEPENDENCE-ADJUSTED MULTIPLE TESTING

Robust likelihood ratio tests for composite nulls and alternatives

A Non-Parametric Box-Cox Approach to Robustifying High-Dimensional Linear Hypothesis Testing

Robust testing in generalized linear models by sign flipping score contributions

On Universality and Training in Binary Hypothesis Testing

Kernel Robust Hypothesis Testing

Hypothesis testing at the extremes: fast and robust association for high-throughput data

Robust Hypothesis Testing Using Wasserstein Uncertainty Sets

Reprint: Hypothesis testing on high dimensional quantile regression

Hypothesis Testing in Gaussian Graphical Models: Novel Goodness-of-Fit Tests and Conditional Randomization Tests

Generalized Likelihood Ratio Test for Adversarially Robust Hypothesis Testing

Generalized Wald Test for Binary Composite Hypothesis Test

A Nonparametric Model Checking Test for Functional Linear Composite Quantile Regression Models

Postselected quantum hypothesis testing

Robust Hypothesis Testing with Wasserstein Uncertainty Sets

Bayesian composite -quantile regression

Power Enhancement in High Dimensional Cross-Sectional Tests