Abstract:In a novel approach to the multiple testing problem, Efron (2004; 2007) formulated estimators of the distribution of test statistics or nominal p-values under a null distribution suitable for modeling the data of thousands of unaffected genes, non-associated single-nucleotide polymorphisms, or other biological features. Estimators of the null distribution can improve not only the empirical Bayes procedure for which it was originally intended, but also many other multiple comparison procedures. Such estimators serve as the groundwork for the proposed multiple comparison procedure based on a recent frequentist method of minimizing posterior expected loss, exemplified with a non-additive loss function designed for genomic screening rather than for validation. The merit of estimating the null distribution is examined from the vantage point of conditional inference in the remainder of the paper. In a simulation study of genome-scale multiple testing, conditioning the observed confidence level on the estimated null distribution as an approximate ancillary statistic markedly improved conditional inference. To enable researchers to determine whether to rely on a particular estimated null distribution for inference or decision making, an information-theoretic score is provided that quantifies the benefit of conditioning. As the sum of the degree of ancillarity and the degree of inferential relevance, the score reflects the balance conditioning would strike between the two conflicting terms. Applications to gene expression microarray data illustrate the methods introduced.

Two New Estimators for the Proportion of True Null Hypotheses in Multiple Test

Multiple-Testing Procedure Based on Generalized Inference of True Null Hypotheses Proportion

Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses

A Change-Point Approach to Estimating the Proportion of False Null Hypotheses in Multiple Testing

Optimal rates of convergence for estimating the null density and proportion of nonnull effects in large-scale multiple testing

Estimation of Proportion of Null Hypotheses Under Dependence

Consistent estimation of the proportion of false nulls and FDR for adaptive multiple testing Normal means under weak dependence

New multiple testing method under no dependency assumption, with application to multiple comparisons problem

Optimal False Discovery Rate Control for Large Scale Multiple Testing with Auxiliary Information

Generalized FWER Control Procedures for Testing Multiple Hypotheses

Statistical inference on mixing proportion

Pretest estimation in combining probability and non-probability samples

A Modified Combined P-Value Multiple Test

Approximating Null Distribution of the M-Test Statistics in Linear Models

Analysis of error control in large scale two-stage multiple hypothesis testing

GMM-based Procedure for Multiple Hypotheses Testing

Distribution-free Multiple Testing

Estimating false discovery proportion in multiple comparison under dependency.

Estimating the null distribution for conditional inference and genome-scale screening

Two New Methods for Non-Inferiority Testing of the Ratio in Matched-Pair Setting

Estimating the number and effect sizes of non-null hypotheses