Abstract:We propose a general and flexible procedure for testing multiple hypotheses about sequential (or streaming) data that simultaneously controls both the false discovery rate (FDR) and false nondiscovery rate (FNR) under minimal assumptions about the data streams, which may differ in distribution and dimension and be dependent. All that is needed is a test statistic for each data stream that controls its conventional type I and II error probabilities, and no information or assumptions are required about the joint distribution of the statistics or data streams. The procedure can be used with sequential, group sequential, truncated, or other sampling schemes. The procedure is a natural extension of Benjamini and Hochberg's (<a href="#">1995</a> Benjamini, Y. and Hochberg, Y. (1995). Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing, Journal of the Royal Statistical Society, Series B: Methodological 57: 289–300. doi:10.1111/j.2517-6161.1995.tb02031.x<a href="/servlet/linkout?suffix=CIT0009&dbid=16&doi=10.1080%2F07474946.2020.1726686&key=10.1111%2Fj.2517-6161.1995.tb02031.x">[Crossref]</a> , <a class="google-scholar" href="http://scholar.google.com/scholar_lookup?hl=en&publication_year=1995&pages=289-300&author=Y.+Benjamini&author=Y.+Hochberg&title=Controlling+the+False+Discovery+Rate%3A+A+Practical+and+Powerful+Approach+to+Multiple+Testing">[Google Scholar]</a>) widely used fixed sample size procedure to the domain of sequential data, with the added benefit of simultaneous FDR and FNR control that sequential sampling affords. We prove the procedure's error control and give some tips for implementation in commonly encountered testing situations.

Sequential tests of multiple hypotheses controlling type I and II familywise error rates

A Rejection Principle for Sequential Tests of Multiple Hypotheses Controlling Familywise Error Rates

Sequential tests of multiple hypotheses controlling false discovery and nondiscovery rates

Efficient three-stage $t$-tests

Generalizing Simes' test and Hochberg's stepup procedure

The sequential rejection principle of familywise error control

Familywise Error Rate Controlling Procedures for Discrete Data

Power-enhanced multiple decision functions controlling family-wise error and false discovery rates

On Asymptotic Behaviors of Stepwise Multiple Testing Procedures

On limiting behaviors of stepwise multiple testing procedures

Analysis of error control in large scale two-stage multiple hypothesis testing

Adjusted inference for multiple testing procedure in group sequential designs

Efficiency in sequential testing: Comparing the sequential probability ratio test and the sequential Bayes factor test

Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing

Large-scale adaptive multiple testing for sequential data controlling false discovery and nondiscovery rates

A Family of Bayes Multiple Testing Procedures

Private Sequential Hypothesis Testing for Statisticians: Privacy, Error Rates, and Sample Size

Sequential Tests of Statistical Hypotheses with Confidence Limits

Familywise error for multiple time‐to‐event endpoints in a group sequential design

Generalized Sequential Probability Ratio Test for Separate Families of Hypotheses.

Robustness of multiple testing procedures against dependence