Abstract:Consider a binary statistical hypothesis testing problem, where $n$ independent and identically distributed random variables $Z^n$ are either distributed according to the null hypothesis $P$ or the alternative hypothesis $Q$, and only $P$ is known. A well-known test that is suitable for this case is the so-called Hoeffding test, which accepts $P$ if the Kullback-Leibler (KL) divergence between the empirical distribution of $Z^n$ and $P$ is below some threshold. This work characterizes the first and second-order terms of the type-II error probability for a fixed type-I error probability for the Hoeffding test as well as for divergence tests, where the KL divergence is replaced by a general divergence. It is demonstrated that, irrespective of the divergence, divergence tests achieve the first-order term of the Neyman-Pearson test, which is the optimal test when both $P$ and $Q$ are known. In contrast, the second-order term of divergence tests is strictly worse than that of the Neyman-Pearson test. It is further demonstrated that divergence tests with an invariant divergence achieve the same second-order term as the Hoeffding test, but divergence tests with a non-invariant divergence may outperform the Hoeffding test for some alternative hypotheses $Q$. Potentially, this behavior could be exploited by a composite hypothesis test with partial knowledge of the alternative hypothesis $Q$ by tailoring the divergence of the divergence test to the set of possible alternative hypotheses.

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper aims to study the second-order asymptotic performance of the Hoeffding test and other divergence tests in binary hypothesis testing. Specifically, the paper focuses on the first-order and second-order terms of the type II error probability for the Hoeffding test and general divergence tests when the type I error probability is fixed. #### Main Contributions: 1. **Characterization of First-Order and Second-Order Terms**: - The paper characterizes the first-order and second-order terms of the divergence test for any divergence (Theorem 1). The results show that regardless of the choice of divergence, the divergence test can achieve the same first-order term as the Neyman-Pearson test. - For the Hoeffding test (i.e., the case of KL divergence), the paper derives the specific expression for its second-order term and demonstrates that it is inferior to the second-order term of the Neyman-Pearson test. 2. **Comparison of Invariant and Non-Invariant Divergences**: - The paper shows that when the divergence is an invariant divergence, the divergence test can achieve the same second-order term as the Hoeffding test. - When the divergence is a non-invariant divergence, its second-order term may be better or worse than the Hoeffding test, depending on the alternative hypothesis. 3. **Numerical Results**: - In the fourth section, the paper compares the second-order terms of the Hoeffding test and non-invariant divergence tests through numerical experiments, finding that for certain alternative hypotheses, the second-order term of the non-invariant divergence test may be superior to that of the Hoeffding test. ### Conclusion By analyzing the second-order asymptotic performance of the Hoeffding test and other divergence tests, the paper reveals the impact of different divergence choices on test performance and provides theoretical guidance for practical applications. In particular, for cases with partially known alternative hypotheses, the test performance can be optimized by choosing an appropriate divergence.

On the Second-Order Asymptotics of the Hoeffding Test and Other Divergence Tests

Detecting abnormal situations using the Kullback-Leibler divergence.

Universal and Composite Hypothesis Testing via Mismatched Divergence

A Hyperbolic Divergence Based Nonparametric Test for Two‐sample Multivariate Distributions

Statistical divergences in high-dimensional hypothesis testing and a modern technique for estimating them

On Likelihood Functions to Minimize KL Divergence in Binary Hypothesis Testing.

Robust Kullback-Leibler Divergence and Universal Hypothesis Testing for Continuous Distributions

Second Order Asymptotics for Quantum Hypothesis Testing

Central Limit Theorem on Symmetric Kullback-Leibler (KL) Divergence

The Logarithmic Super Divergence and its use in Statistical Inference

Asymptotics for Outlier Hypothesis Testing

Sub-Gaussian Error Bounds for Hypothesis Testing

A High-dimensional Convergence Theorem for U-statistics with Applications to Kernel-based Testing

On divergences tests for composite hypotheses under composite likelihood

A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations

Universal Hypothesis Testing with Kernels: Asymptotically Optimal Tests for Goodness of Fit

Asymptotic Normality and Efficiency of Certain Nonparametric Test Statistics

A Cramér moderate deviation theorem for Hotelling's $T^2$-statistic with applications to global tests

A Cramér Moderate Deviation Theorem for Hotelling’s $T^{2}$-Statistic with Applications to Global Tests

The Sample Complexity of Simple Binary Hypothesis Testing

Testing the Isotropic Cauchy Hypothesis