Abstract:In modern scientific experiments, we frequently encounter data that have large dimensions, and in some experiments, such high dimensional data arrive sequentially rather than full data being available all at a time. We develop multiple testing procedures with simultaneous control of false discovery and nondiscovery rates when $m$-variate data vectors $\mathbf{X}_1, \mathbf{X}_2, \dots$ are observed sequentially or in groups and each coordinate of these vectors leads to a hypothesis testing. Existing multiple testing methods for sequential data uses fixed stopping boundaries that do not depend on sample size, and hence, are quite conservative when the number of hypotheses $m$ is large. We propose sequential tests based on adaptive stopping boundaries that ensure shrinkage of the continue sampling region as the sample size increases. Under minimal assumptions on the data sequence, we first develop a test based on an oracle test statistic such that both false discovery rate (FDR) and false nondiscovery rate (FNR) are nearly equal to some prefixed levels with strong control. Under a two-group mixture model assumption, we propose a data-driven stopping and decision rule based on local false discovery rate statistic that mimics the oracle rule and guarantees simultaneous control of FDR and FNR asymptotically as $m$ tends to infinity. Both the oracle and the data-driven stopping times are shown to be finite (i.e., proper) with probability 1 for all finite $m$ and converge to a finite constant as $m$ grows to infinity. Further, we compare the data-driven test with the existing gap rule proposed in He and Bartroff (2021) and show that the ratio of the expected sample sizes of our method and the gap rule tends to zero as $m$ goes to infinity. Extensive analysis of simulated datasets as well as some real datasets illustrate the superiority of the proposed tests over some existing methods.

Directional false discovery rate control in large-scale multiple comparisons

Optimal False Discovery Rate Control for Large Scale Multiple Testing with Auxiliary Information

False Discovery Control in Multiple Testing: A Brief Overview of Theories and Methodologies

Split Knockoffs for Multiple Comparisons: Controlling the Directional False Discovery Rate

Asymptotic false discovery control of the Benjamini-Hochberg procedure for pairwise comparisons

Exploratory data analysis for large-scale multiple testing problems and its application in gene expression studies

A New Procedure for Controlling False Discovery Rate in Large-Scale t-tests

2dGBH: Two-dimensional group Benjamini–Hochberg procedure for false discovery rate control in two-way multiple testing of genomic data

Large-scale adaptive multiple testing for sequential data controlling false discovery and nondiscovery rates

False discovery rate control: Moving beyond the Benjamini–Hochberg method

Large-scale Multiple Testing: Fundamental Limits of False Discovery Rate Control and Compound Oracle

Directional FDR Control for Sub-Gaussian Sparse GLMs

A practical guide to methods controlling false discoveries in computational biology

Selective testing and its effect on false discovery rate controlling procedures under discrete framework

A two-step strategy for detecting differential gene expression in cDNA microarray data

Controlling the Rate of GWAS False Discoveries

Multiple Testing of Submatrices of a Precision Matrix with Applications to Identification of Between Pathway Interactions

Modifying the false discovery rate procedure based on the information theory under arbitrary correlation structure and its performance in high-dimensional genomic data

False Discovery Rate Control in Cancer Biomarker Selection Using Knockoffs.

Controlling false discovery rate for mediator selection in high-dimensional data

Optimal design for high-throughput screening via false discovery rate control