Abstract:In modern scientific experiments, we frequently encounter data that have large dimensions, and in some experiments, such high dimensional data arrive sequentially rather than full data being available all at a time. We develop multiple testing procedures with simultaneous control of false discovery and nondiscovery rates when $m$-variate data vectors $\mathbf{X}_1, \mathbf{X}_2, \dots$ are observed sequentially or in groups and each coordinate of these vectors leads to a hypothesis testing. Existing multiple testing methods for sequential data uses fixed stopping boundaries that do not depend on sample size, and hence, are quite conservative when the number of hypotheses $m$ is large. We propose sequential tests based on adaptive stopping boundaries that ensure shrinkage of the continue sampling region as the sample size increases. Under minimal assumptions on the data sequence, we first develop a test based on an oracle test statistic such that both false discovery rate (FDR) and false nondiscovery rate (FNR) are nearly equal to some prefixed levels with strong control. Under a two-group mixture model assumption, we propose a data-driven stopping and decision rule based on local false discovery rate statistic that mimics the oracle rule and guarantees simultaneous control of FDR and FNR asymptotically as $m$ tends to infinity. Both the oracle and the data-driven stopping times are shown to be finite (i.e., proper) with probability 1 for all finite $m$ and converge to a finite constant as $m$ grows to infinity. Further, we compare the data-driven test with the existing gap rule proposed in He and Bartroff (2021) and show that the ratio of the expected sample sizes of our method and the gap rule tends to zero as $m$ goes to infinity. Extensive analysis of simulated datasets as well as some real datasets illustrate the superiority of the proposed tests over some existing methods.

Large-scale simultaneous inference under dependence

Simultaneous inference: When should hypothesis testing problems be combined?

Locally Simultaneous Inference

Large-scale adaptive multiple testing for sequential data controlling false discovery and nondiscovery rates

Multiple two-sample testing under arbitrary covariance dependency with an application in imaging mass spectrometry

Simultaneous inference for generalized linear models with unmeasured confounders

Simultaneous high-probability bounds on the false discovery proportion in structured, regression, and online settings

Simultaneous Inference in General Parametric Models

A NEW PERSPECTIVE ON ROBUST M-ESTIMATION: FINITE SAMPLE THEORY AND APPLICATIONS TO DEPENDENCE-ADJUSTED MULTIPLE TESTING

Large-Scale Simultaneous Testing Using Kernel Density Estimation

Estimating false discovery proportion in multiple comparison under dependency.

A Decorrelating and Debiasing Approach to Simultaneous Inference for High-Dimensional Confounded Models

Joint Sequential Detection and Isolation for Dependent Data Streams

Optimal False Discovery Rate Control for Large Scale Multiple Testing with Auxiliary Information

Consistent estimation of the proportion of false nulls and FDR for adaptive multiple testing Normal means under weak dependence

Robustness of multiple testing procedures against dependence

L-C2ST: Local Diagnostics for Posterior Approximations in Simulation-Based Inference

Statistical Inferences for Complex Dependence of Multimodal Imaging Data

Large-scale dependent multiple testing via higher-order hidden Markov models

Non-marginal Decisions: A Novel Bayesian Multiple Testing Procedure

Semi-parametric inference for large-scale data with temporally dependent noise