Abstract:Given a composite null $ \mathcal P$ and composite alternative $ \mathcal Q$, when and how can we construct a p-value whose distribution is exactly uniform under the null, and stochastically smaller than uniform under the alternative? Similarly, when and how can we construct an e-value whose expectation exactly equals one under the null, but its expected logarithm under the alternative is positive? We answer these basic questions, and other related ones, when $ \mathcal P$ and $ \mathcal Q$ are convex polytopes (in the space of probability measures). We prove that such constructions are possible if and only if $ \mathcal Q$ does not intersect the span of $ \mathcal P$. If the p-value is allowed to be stochastically larger than uniform under $P\in \mathcal P$, and the e-value can have expectation at most one under $P\in \mathcal P$, then it is achievable whenever $ \mathcal P$ and $ \mathcal Q$ are disjoint. More generally, even when $ \mathcal P$ and $ \mathcal Q$ are not polytopes, we characterize the existence of a bounded nontrivial e-variable whose expectation exactly equals one under any $P \in \mathcal P$. The proofs utilize recently developed techniques in simultaneous optimal transport. A key role is played by coarsening the filtration: sometimes, no such p-value or e-value exists in the richest data filtration, but it does exist in some reduced filtration, and our work provides the first general characterization of this phenomenon. We also provide an iterative construction that explicitly constructs such processes, and under certain conditions it finds the one that grows fastest under a specific alternative $Q$. We discuss implications for the construction of composite nonnegative (super)martingales, and end with some conjectures and open problems.

A Fiducial Argument for Generalized P-Value

p-Value as the Strength of Evidence Measured by Confidence Distribution

Fiducialize statistical significance: transforming p-values into conservative posterior probabilities and Bayes factors

Dempster-Shafer P-values: Thoughts on an Alternative Approach for Multinomial Inference

A minimum Wasserstein distance approach to Fisher's combination of independent discrete p-values

Extended MinP Tests for Global and Multiple testing

A Bernstein-von Mises Theorem for Generalized Fiducial Distributions

A measure of evidence based on the likelihood-ratio statistics

Multiple testing of composite null hypotheses for discrete data using randomized $p$-values

P-value: A Bless or A Curse for Evidence-Based Studies?

Decision theory via model-free generalized fiducial inference

On two-sided p-values for non-symmetric distributions

Combined p-value functions for meta-analysis

Accurate $p$-Value Calculation for Generalized Fisher's Combination Tests Under Dependence

Selective inference is easier with p-values

Generalizing Simes' test and Hochberg's stepup procedure

On the P-value for Members of the Cressie-Read Family of Divergence Statistics

Confidence distributions and hypothesis testing

Efficient likelihood-based inference for the generalized Pareto distribution

On the existence of powerful p-values and e-values for composite hypotheses

Generalized fiducial factor: an alternative to the Bayes factor for forensic identification of source problems