Abstract:The test statistics underpinning several methods for combining p -values are special cases of generalized mean p -value (GMP), including the minimum (Bonferroni procedure), harmonic mean and geometric mean. A key assumption influencing the practical performance of such methods concerns the dependence between p -values. Approaches that do not require specific knowledge of the dependence structure are practically convenient. Vovk and Wang derived significance thresholds for GMPs under the worst-case scenario of arbitrary dependence using results from Robust Risk Analysis (RRA). Here I calculate significance thresholds and closed testing procedures using Generalized Central Limit Theorem (GCLT). GCLT formally assumes independence, but enjoys a degree of robustness to dependence. The GCLT thresholds are less stringent than RRA thresholds, with the disparity increasing as the exponent of the GMP ( r ) increases. I motivate a model of p -value dependence based on a Wishart-Multivariate-Gamma distribution for the underlying log-likelihood ratios. In simulations under this model, the RRA thresholds produced tests that were usually less powerful than Bonferroni, while the GCLT thresholds produced tests more powerful than Bonferroni, for all r > − ∞. Above r > − 1, the GCLT thresholds suffered pronounced false positive rates. Above r > − 1/2, standard central limit theorem applied and the GCLT thresholds no longer possessed any useful robustness to dependence. I consider the implications of these results in the context of various interpretations of GMPs, and conclude that the GCLT-based harmonic mean p -value procedure and Simes' (1986) test represent good compromises in power-robustness trade-off for combining dependent tests.

A Modified Combined P-Value Multiple Test

Multiple testing using uniform filtering of ordered p-values

A New Multiple Testing Method in the Dependent Case

Effective Positive Cauchy Combination Test

New multiple testing method under no dependency assumption, with application to multiple comparisons problem

Multiple testing of composite null hypotheses for discrete data using randomized $p$-values

A Combined P -Value Test for the Mean Difference of High-Dimensional Data

Weighted p -value procedures for controlling FDR of grouped hypotheses

On P-value Combination Procedures

The Lévy combination test

Combined p-value functions for meta-analysis

Multiple testing of interval composite null hypotheses using randomized p-values

Accurate $p$-Value Calculation for Generalized Fisher's Combination Tests Under Dependence

A New Generalized P-Value Approach for Testing Equality of Coefficients of Variation in K Normal Populations

Extended MinP Tests for Global and Multiple testing

Generalized mean p-values for combining dependent tests: comparison of generalized central limit theorem and robust risk analysis

Cauchy combination test: a powerful test with analytic p-value calculation under arbitrary dependency structures

Two New Estimators for the Proportion of True Null Hypotheses in Multiple Test

Multiple testing when many $p$-values are uniformly conservative, with application to testing qualitative interaction in educational interventions

A Unified and Optimal Multiple Testing Framework based on rho-values

Generalized FWER Control Procedures for Testing Multiple Hypotheses