Abstract:We propose a coupled bootstrap (CB) method for the test error of an arbitrary algorithm that estimates the mean in a Poisson sequence, often called the Poisson means problem. The idea behind our method is to generate two carefully-designed data vectors from the original data vector, by using synthetic binomial noise. One such vector acts as the training sample and the second acts as the test sample. To stabilize the test error estimate, we average this over multiple bootstrap B of the synthetic noise. A key property of the CB estimator is that it is unbiased for the test error in a Poisson problem where the original mean has been shrunken by a small factor, driven by the success probability $p$ in the binomial noise. Further, in the limit as $B \to \infty$ and $p \to 0$, we show that the CB estimator recovers a known unbiased estimator for test error based on Hudson's lemma, under no assumptions on the given algorithm for estimating the mean (in particular, no smoothness assumptions). Our methodology applies to two central loss functions that can be used to define test error: Poisson deviance and squared loss. Via a bias-variance decomposition, for each loss function, we analyze the effects of the binomial success probability and the number of bootstrap samples and on the accuracy of the estimator. We also investigate our method empirically across a variety of settings, using simulated as well as real data.

Bootstrap of means under stratified sampling

Cluster-Robust Bootstrap Inference in Quantile Regression Models

Empirical saddlepoint approximations of the Studentized mean under stratified random sampling

Bootstrap Inference for Panel Data Quantile Regression

Bootstrap algorithms for small samples

Wild bootstrap inference for penalized quantile regression for longitudinal data

The Big Data Bootstrap

The resampling method via representative points

On the implementation of stratified two-stage simple random sampling without replacement, with possible collapsed strata

Sharp variance estimator and causal bootstrap in stratified randomized experiments

Unbiased Test Error Estimation in the Poisson Means Problem via Coupled Bootstrap Techniques

Deterministic bootstrapping for a class of bootstrap methods

Scalable Nonparametric Sampling from Multimodal Posteriors with the Posterior Bootstrap

Optimal Subsampling Bootstrap for Massive Data

Bootstrap Resampling Approaches for Repeated Measure Designs: Relative Robustness to Sphericity and Normality Violations

On the Assessment of Bootstrap Intervals for Samples of Fixed Size

Rescaling bootstrap variance estimation technique under dual frame surveys with unknown domain sizes

A Simple Bootstrap for Chatterjee's Rank Correlation

Bootstrap estimation of the proportion of outliers in robust regression

Bootstrap for inference after model selection and model averaging for likelihood models

Strong uniform laws of large numbers for bootstrap means and other randomly weighted sums