Abstract:The false coverage rate (FCR) is the expected ratio of number of constructed confidence intervals (CIs) that fail to cover their respective parameters to the total number of constructed CIs. Procedures for FCR control exist in the offline setting, but none so far have been designed with the online setting in mind. In the online setting, there is an infinite sequence of fixed unknown parameters $\theta_t$ ordered by time. At each step, we see independent data that is informative about $\theta_t$, and must immediately make a decision whether to report a CI for $\theta_t$ or not. If $\theta_t$ is selected for coverage, the task is to determine how to construct a CI for $\theta_t$ such that $\text{FCR} \leq \alpha$ for any $T\in \mathbb{N}$. A straightforward solution is to construct at each step a $(1-\alpha)$ level conditional CI. In this paper, we present a novel solution to the problem inspired by online false discovery rate (FDR) algorithms, which only requires the statistician to be able to construct a marginal CI at any given level. Apart from the fact that marginal CIs are usually simpler to construct than conditional ones, the marginal procedure has an important qualitative advantage over the conditional solution, namely, it allows selection to be determined by the candidate CI itself. We take advantage of this to offer solutions to some online problems which have not been addressed before. For example, we show that our general CI procedure can be used to devise online sign-classification procedures that control the false sign rate (FSR). In terms of power and length of the constructed CIs, we demonstrate that the two approaches have complementary strengths and weaknesses using simulations. Last, all of our methodology applies equally well to online FCR control for prediction intervals, having particular implications for assumption-free selective conformal inference.

False Discovery Rate Control for Confounder Selection Using Mirror Statistics

Local False Discovery Rate Estimation with Competition-Based Procedures for Variable Selection

High-Dimensional False Discovery Rate Control for Dependent Variables

Controlling False Discovery Rates under Cross-Sectional Correlations

2dFDR: a new approach to confounder adjustment substantially increases detection power in omics association studies

Controlling false discovery rate for mediator selection in high-dimensional data

The Conditional Prediction Function: A Novel Technique to Control False Discovery Rate for Complex Models

False Discovery Control in Multiple Testing: A Brief Overview of Theories and Methodologies

Null-free False Discovery Rate Control Using Decoy Permutations

Controlling the False Discovery Rate in Subspace Selection

False Variable Selection Rates in Regression

Optimal control of false discovery criteria in the two-group model

Controlling the False Discovery Rate for Binary Feature Selection via Knockoff

Evaluating FDR and stratified FDR control approaches for high-throughput biological studies

Controlling the Rate of GWAS False Discoveries

Controlled Variable Selection from Summary Statistics Only? A Solution via GhostKnockoffs and Penalized Regression

Online Control of the False Coverage Rate and False Sign Rate

False discovery rate control: Moving beyond the Benjamini–Hochberg method

False Discovery Rate Control in Cancer Biomarker Selection Using Knockoffs.

Online control of the false discovery rate in biomedical research

Optimal False Discovery Rate Control for Large Scale Multiple Testing with Auxiliary Information