Growth-Optimal E-Variables and an extension to the multivariate Csiszár-Sanov-Chernoff Theorem

Peter Grünwald,Yunda Hao,Akshay Balsubramani
2024-12-23
Abstract:We consider growth-optimal e-variables with maximal e-power, both in an absolute and relative sense, for simple null hypotheses for a $d$-dimensional random vector, and multivariate composite alternatives represented as a set of $d$-dimensional means $\meanspace_1$. These include, among others, the set of all distributions with mean in $\meanspace_1$, and the exponential family generated by the null restricted to means in $\meanspace_1$. We show how these optimal e-variables are related to Csiszár-Sanov-Chernoff bounds, first for the case that $\meanspace_1$ is convex (these results are not new; we merely reformulate them) and then for the case that $\meanspace_1$ `surrounds' the null hypothesis (these results are new).
Information Theory,Statistics Theory
What problem does this paper attempt to address?
### What problems does this paper attempt to solve? This paper mainly explores how to construct optimal e - variables (growth - optimal e - variables) in multiple - hypothesis testing, especially when the null hypothesis is a simple hypothesis and the alternative hypothesis is a composite hypothesis. Specifically, the article addresses the following issues: 1. **Defining and understanding growth - optimal e - variables**: The article studies how to define absolute and relative growth - optimal e - variables (GROW e - variables) for a d - dimensional random vector \(Y\), under a given simple null hypothesis \(P_0\) and a multivariate composite alternative hypothesis \(H_1\). These e - variables have an optimal growth rate in the worst - case scenario. 2. **Extension of the Csiszár - Sanov - Chernoff (CSC) inequality**: The article shows the relationship between these optimal e - variables and the Csiszár - Sanov - Chernoff (CSC) bound, and extends this bound to the case of non - convex sets \(M_1\), especially when \(M_1\) encloses the null hypothesis. 3. **Handling \(M_1\) that encloses the null hypothesis**: The article further considers the case where the complement of \(M_1\) is connected, bounded, and contains the zero point. This setting is closer to practical applications and is more relevant to the multivariate central limit theorem (CLT). The author proposes two methods for extending GROW e - variables: absolute extension and relative optimal extension, and proves a new CSC bound. 4. **Asymptotic analysis and model complexity**: Based on the maximum - likelihood estimation (MLE), the article provides an asymptotic expression for the minimax regret term \(mmreg\), and relates it to the BIC/MDL model complexity. This helps to understand the influence of the boundary \(M_1\) and gives an asymptotic expression for the absolute GROW e - variable. ### Specific problem summary - **Objective**: Construct optimal e - variables for statistical inference in multiple - hypothesis testing. - **Method**: Derive the form of the optimal e - variables by using Csiszár - Topsøe's Pythagorean theorem and information projection theory, and extend the CSC bound to more general cases. - **Application scenario**: Pay special attention to the case where \(M_1\) encloses the null hypothesis, which is a common setting in many practical problems. ### Mathematical formula representation - **KL divergence**: \[ D(P \| Q)=\mathbb{E}_P\left[\log \frac{p(Y)}{q(Y)}\right] \] - **GROW e - variable**: \[ S_{\text{grow}}=\frac{\bar{p}_{\mu^*}(Y)}{p_0(Y)} \] where \(\mu^*\) is the mean parameter that minimizes \(D(\bar{P}_{\mu} \| P_0)\). - **CSC bound**: \[ P_0(Y \in M_1) \leq e^{-D} \] where \(D = \inf_{\mu \in M_1} D(\bar{P}_{\mu} \| P_0)\). These formulas show how to construct optimal e - variables by optimizing KL divergence and evaluate their performance through the CSC bound.