Abstract:$P$-values that are derived from continuously distributed test statistics are typically uniformly distributed on $(0,1)$ under least favorable parameter configurations (LFCs) in the null hypothesis. Conservativeness of a $p$-value $P$ (meaning that $P$ is under the null hypothesis stochastically larger than a random variable which is uniformly distributed on $(0,1)$) can occur if the test statistic from which $P$ is derived is discrete, or if the true parameter value under the null is not an LFC. To deal with both of these sources of conservativeness, we present two approaches utilizing randomized $p$-values, namely single-stage and two-stage randomization. We illustrate their effectiveness for testing a composite null hypothesis under a binomial model. We also give an example of how the proposed $p$-values can be used to test a composite null in group testing designs. Similar to previous findings, we find that the proposed randomized $p$-values are less conservative compared to non-randomized $p$-values under the null hypothesis, but that they are stochastically not smaller under the alternative. The problem of establishing the validity of randomized $p$-values is not trivial and has received attention in previous literature. We show that our proposed randomized $p$-values are valid under various discrete statistical models which are such that the distribution of the corresponding test statistic belongs to an exponential family. The behaviour of the power function for the tests based on the proposed randomized $p$-values as a function of the sample size is also investigated. Simulations and a real data analysis are used to compare the different considered $p$-values.

Extended MinP Tests for Global and Multiple testing

Efficient three-stage $t$-tests

M-Testing Using Finite and Infinite Dimensional Parameter Estimators

A New Multiple Testing Method in the Dependent Case

Enhanced power enhancements for testing many moment equalities: Beyond the $2$- and $\infty$-norm

Sequential Cauchy Combination Test for Multiple Testing Problems with Financial Applications

Characteristic Function-Based Testing for Multifactor Continuous-Time Markov Models Via Nonparametric Regression

Testing in the Presence of Nuisance Parameters: Some Comments on Tests Post-Model-Selection and Random Critical Values

A NEW PERSPECTIVE ON ROBUST M-ESTIMATION: FINITE SAMPLE THEORY AND APPLICATIONS TO DEPENDENCE-ADJUSTED MULTIPLE TESTING

Multiple testing when many $p$-values are uniformly conservative, with application to testing qualitative interaction in educational interventions

QuickMMCTest - Quick Multiple Monte Carlo Testing

The Lévy combination test

New Approaches for Testing Slope Homogeneity in Large Panel Data Models

Optimal testing using combined test statistics across independent studies

Optimal exact tests for multiple binary endpoints

A minimum Wasserstein distance approach to Fisher's combination of independent discrete p-values

Multiple testing of composite null hypotheses for discrete data using randomized $p$-values

RMST‐based multiple contrast tests in general factorial designs

Multiple testing of interval composite null hypotheses using randomized p-values

RMST-based multiple contrast tests in general factorial designs

Multiple testing using uniform filtering of ordered p-values