Abstract:We propose a general and flexible procedure for testing multiple hypotheses about sequential (or streaming) data that simultaneously controls both the false discovery rate (FDR) and false nondiscovery rate (FNR) under minimal assumptions about the data streams, which may differ in distribution and dimension and be dependent. All that is needed is a test statistic for each data stream that controls its conventional type I and II error probabilities, and no information or assumptions are required about the joint distribution of the statistics or data streams. The procedure can be used with sequential, group sequential, truncated, or other sampling schemes. The procedure is a natural extension of Benjamini and Hochberg's (<a href="#">1995</a> Benjamini, Y. and Hochberg, Y. (1995). Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing, Journal of the Royal Statistical Society, Series B: Methodological 57: 289–300. doi:10.1111/j.2517-6161.1995.tb02031.x<a href="/servlet/linkout?suffix=CIT0009&dbid=16&doi=10.1080%2F07474946.2020.1726686&key=10.1111%2Fj.2517-6161.1995.tb02031.x">[Crossref]</a> , <a class="google-scholar" href="http://scholar.google.com/scholar_lookup?hl=en&publication_year=1995&pages=289-300&author=Y.+Benjamini&author=Y.+Hochberg&title=Controlling+the+False+Discovery+Rate%3A+A+Practical+and+Powerful+Approach+to+Multiple+Testing">[Google Scholar]</a>) widely used fixed sample size procedure to the domain of sequential data, with the added benefit of simultaneous FDR and FNR control that sequential sampling affords. We prove the procedure's error control and give some tips for implementation in commonly encountered testing situations.

Fdr Control for Pairwise Comparisons of Normal Means Based on Goodness of Fit Test

A New Procedure for Controlling False Discovery Rate in Large-Scale t-tests

On Controlling the False Discovery Rate in Multiple Testing of the Means of Correlated Normals Against Two-Sided Alternatives

Consistent estimation of the proportion of false nulls and FDR for adaptive multiple testing Normal means under weak dependence

False Discovery Control in Multiple Testing: A Brief Overview of Theories and Methodologies

Asymptotic false discovery control of the Benjamini-Hochberg procedure for pairwise comparisons

On stepdown control of the false discovery proportion

A Comparative Review of Methods for Comparing Means Using Partially Paired Data

A step-down multiple hypotheses testing procedure that controls the false discovery rate under independence

A Multiple Comparisons Procedure for Detecting Differences Between Treatments and A Control in Two-Factor Experiments

Two-stage Hypothesis Tests for Variable Interactions with FDR Control

Multiple Comparisons of Treatment Against Control under Unequal Variances Using Parametric Bootstrap

Directional FDR Control for Sub-Gaussian Sparse GLMs

Familywise Error Rate Controlling Procedures for Discrete Data

Null-free False Discovery Rate Control Using Decoy Permutations

Analysis of small sample size studies using nonparametric bootstrap test with pooled resampling method

Heterocedasticity-Adjusted Ranking and Thresholding for Large-Scale Multiple Testing

Sequential tests of multiple hypotheses controlling false discovery and nondiscovery rates

A New Multiple Testing Method in the Dependent Case

A grouped, selectively weighted false discovery rate procedure

Power-enhanced multiple decision functions controlling family-wise error and false discovery rates