Abstract:Data with a large p (number of covariates) and/or a large n (sample size) are now commonly encountered. For many problems, regularization especially penalization is adopted for estimation and variable selection. The straightforward application of penalization to large datasets demands a "big computer" with high computational power. To improve computational feasibility, we develop bootstrap penalization, which dissects a big penalized estimation into a set of small ones, which can be executed in a highly parallel manner and each only demands a "small computer". The proposed approach takes different strategies for data with different characteristics. For data with a large p but a small to moderate n, covariates are first clustered into relatively homogeneous blocks. The proposed approach consists of two sequential steps. In each step and for each bootstrap sample, we select blocks of covariates and run penalization. The results from multiple bootstrap samples are pooled to generate the final estimate. For data with a large n but a small to moderate p, we bootstrap a small number of subjects, apply penalized estimation, and then conduct a weighted average over multiple bootstrap samples. For data with a large p and a large n, the natural marriage of the previous two methods is applied. Numerical studies, including simulations and data analysis, show that the proposed approach has computational and numerical advantages over the straightforward application of penalization. An R package has been developed to implement the proposed methods.

A scalable bootstrap for massive data

The big data bootstrap

A Subsampled Double Bootstrap for Massive Data

Optimal Subsampling Bootstrap for Massive Data

Scalable bootstrap clustering for massive data

Variable Selection with Scalable Bootstrap in Generalized Linear Model for Massive Data

Distributed Bootstrap Simultaneous Inference for High-Dimensional Quantile Regression

Cluster-Robust Bootstrap Inference in Quantile Regression Models

Analyzing large datasets with bootstrap penalization.

Bootstrap algorithms for small samples

Hyperparameter Selection for Subsampling Bootstraps

Parallel Optimisation of Bootstrapping in R

An introduction to the bootstrap

Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy

An Online Bootstrap for Time Series

The Analytical Bootstrap: A New Method For Fast Error Estimation In Approximate Query Processing

Scalable subsampling: computation, aggregation and inference

An adaptively resized parametric bootstrap for inference in high-dimensional generalized linear models

Bootstrap Implementation For Statistical Analysis In Finance With R

A Fast Bootstrap Algorithm for Causal Inference with Large Data