Estimating the variability of the Stein estimator by bootstrap

Gang Yi

DOI: https://doi.org/10.1016/0165-1765(91)90225-A

IF: 1.469

1991-01-01

Economics Letters

Abstract:Theoretical and empirical results indicate that the bootstrap method provides a viable alternative in estimating the scale parameters of Stein-like estimators.

What problem does this paper attempt to address?

Cluster-Robust Bootstrap Inference in Quantile Regression Models

Andreas Hagemann

DOI: https://doi.org/10.1080/01621459.2016.1148610

IF: 4.369

2017-01-02

Journal of the American Statistical Association

Abstract:In this article I develop a wild bootstrap procedure for cluster-robust inference in linear quantile regression models. I show that the bootstrap leads to asymptotically valid inference on the entire quantile regression process in a setting with a large number of small, heterogeneous clusters and provides consistent estimates of the asymptotic covariance function of that process. The proposed bootstrap procedure is easy to implement and performs well even when the number of clusters is much smaller than the sample size. An application to Project STAR data is provided. Supplementary materials for this article are available online.

statistics & probability
Wild bootstrap inference for penalized quantile regression for longitudinal data

Carlos Lamarche,Thomas Parker

DOI: https://doi.org/10.1016/j.jeconom.2022.11.011

IF: 3.363

2023-02-04

Journal of Econometrics

Abstract:The existing theory of penalized quantile regression for longitudinal data has focused primarily on point estimation. In this work, we investigate statistical inference. We propose a wild residual bootstrap procedure and show that it is asymptotically valid for approximating the distribution of the penalized estimator. The model puts no restrictions on individual effects, and the estimator achieves consistency by letting the shrinkage decay in importance asymptotically. The new method is easy to implement and simulation studies show that it has accurate small sample behavior in comparison with existing procedures. Finally, we illustrate the new approach using U.S. Census data to estimate a model that includes more than eighty thousand parameters.

economics,social sciences, mathematical methods,mathematics, interdisciplinary applications
Data Sharpening Method in Regression Confidence Band

Xuyang He,Yuexiang Jiang

DOI: https://doi.org/10.1080/03610926.2020.1760887

2022-01-01

Abstract:The confidence band of functions is complicated by the over-smoothing problem and the residual distribution. In this paper, we use bootstrap and data-sharpening methods to establish a general confidence band. The construction is simple and the band is narrower than existing estimation methods. At the same time, a technique based on quantiles makes the confidence band more controllable and damps down the stochastic error term. Afterwards, we conduct a limited simulation to illustrate that the proposed band performs better than existing ones. Finally, we show the theoretical properties of the results and prove them.
Bootstrap Inference for Panel Data Quantile Regression

Antonio F. Galvao,Thomas Parker,Zhijie Xiao

DOI: https://doi.org/10.1080/07350015.2023.2210189

2023-06-04

Journal of Business and Economic Statistics

Abstract:This article develops bootstrap methods for practical statistical inference in panel data quantile regression models with fixed effects. We consider random-weighted bootstrap resampling and formally establish its validity for asymptotic inference. The bootstrap algorithm is simple to implement in practice by using a weighted quantile regression estimation for fixed effects panel data. We provide results under conditions that allow for temporal dependence of observations within individuals, thus, encompassing a large class of possible empirical applications. Monte Carlo simulations provide numerical evidence the proposed bootstrap methods have correct finite sample properties. Finally, we provide an empirical illustration using the environmental Kuznets curve.

statistics & probability,social sciences, mathematical methods,economics
The Grid Bootstrap for Continuous Time Models

Yiu Lim Lui,Weilin Xiao,Jun Yu

DOI: https://doi.org/10.1080/07350015.2021.1930014

2021-01-01

Journal of Business and Economic Statistics

Abstract:This article proposes the new grid bootstrap to construct confidence intervals (CI) for the persistence parameter in a class of continuous-time models. It is different from the standard grid bootstrap of Hansen in dealing with the initial condition. The asymptotic validity of the CI is discussed under the in-fill scheme. The modified grid bootstrap leads to uniform inferences on the persistence parameter. Its improvement over in-fill asymptotics is achieved by expanding the coefficient-based statistic around its in-fill asymptotic distribution that is non-pivotal and depends on the initial condition. Monte Carlo studies show that the modified grid bootstrap performs better than Hansen's grid bootstrap. Empirical applications to the U.S. interest rates and volatilities suggest significant differences between the two bootstrap procedures when the initial condition is large.
Bootstrap SGD: Algorithmic Stability and Robustness

Andreas Christmann,Yunwen Lei

2024-09-02

Abstract:In this paper some methods to use the empirical bootstrap approach for stochastic gradient descent (SGD) to minimize the empirical risk over a separable Hilbert space are investigated from the view point of algorithmic stability and statistical robustness. The first two types of approaches are based on averages and are investigated from a theoretical point of view. A generalization analysis for bootstrap SGD of Type 1 and Type 2 based on algorithmic stability is done. Another type of bootstrap SGD is proposed to demonstrate that it is possible to construct purely distribution-free pointwise confidence intervals of the median curve using bootstrap SGD.

Machine Learning,Artificial Intelligence
Bootstrapping non-stationary stochastic volatility

H. Peter Boswijk,Giuseppe Cavaliere,Iliyan Georgiev,Anders Rahbek

DOI: https://doi.org/10.1016/j.jeconom.2021.01.005

IF: 3.363

2021-09-01

Journal of Econometrics

Abstract:<p>In this paper we investigate to what extent the bootstrap can be applied to conditional mean models, such as regression or time series models, when the volatility of the innovations is random and possibly non-stationary. In fact, the volatility of many economic and financial time series displays persistent changes and possible non-stationarity. However, the theory of the bootstrap for such models has focused on deterministic changes of the unconditional variance and little is known about the performance and the validity of the bootstrap when the volatility is driven by a non-stationary stochastic process. This includes near-integrated exogenous volatility processes as well as near-integrated GARCH processes, where the conditional variance has a diffusion limit; a further important example is the case where volatility exhibits infrequent jumps. This paper fills this gap in the literature by developing conditions for bootstrap validity in time series and regression models with non-stationary, stochastic volatility. We show that in such cases the distribution of bootstrap statistics (conditional on the data) is random in the limit. Consequently, the conventional approaches to proofs of bootstrap consistency, based on the notion of weak convergence in probability of the bootstrap statistic, fail to deliver the required validity results. Instead, we use the concept of 'weak convergence in distribution' to develop and establish novel conditions for validity of the wild bootstrap, conditional on the volatility process. We apply our results to several testing problems in the presence of non-stationary stochastic volatility, including testing in a location model, testing for structural change using CUSUM-type functionals, and testing for a unit root in autoregressive models. Importantly, we work under sufficient conditions for bootstrap validity that include the absence of statistical leverage effects, i.e., correlation between the error process and its future conditional variance. The results of the paper are illustrated using Monte Carlo simulations, which indicate that a wild bootstrap approach leads to size control even in small samples.</p>

economics,social sciences, mathematical methods,mathematics, interdisciplinary applications
Bootstrapping the Cross-Validation Estimate

Bryan Cai,Fabio Pellegrini,Menglan Pang,Carl de Moor,Changyu Shen,Vivek Charu,Lu Tian

DOI: https://doi.org/10.48550/arXiv.2307.00260

2023-07-01

Methodology

Abstract:Cross-validation is a widely used technique for evaluating the performance of prediction models. It helps avoid the optimism bias in error estimates, which can be significant for models built using complex statistical learning algorithms. However, since the cross-validation estimate is a random value dependent on observed data, it is essential to accurately quantify the uncertainty associated with the estimate. This is especially important when comparing the performance of two models using cross-validation, as one must determine whether differences in error estimates are a result of chance fluctuations. Although various methods have been developed for making inferences on cross-validation estimates, they often have many limitations, such as stringent model assumptions This paper proposes a fast bootstrap method that quickly estimates the standard error of the cross-validation estimate and produces valid confidence intervals for a population parameter measuring average model performance. Our method overcomes the computational challenge inherent in bootstrapping the cross-validation estimate by estimating the variance component within a random effects model. It is just as flexible as the cross-validation procedure itself. To showcase the effectiveness of our approach, we employ comprehensive simulations and real data analysis across three diverse applications.
Bootstrap estimation of the proportion of outliers in robust regression

Qiang Heng,Kenneth Lange

DOI: https://doi.org/10.1007/s11222-024-10526-1

IF: 2.3241

2024-11-23

Statistics and Computing

Abstract:This paper presents a nonparametric bootstrap method for estimating the proportions of inliers and outliers in robust regression models. Our approach is based on the concept of stability, providing robustness against distributional assumptions and eliminating the need for pre-specified confidence levels. Through numerical experiments, we demonstrate that this method yields more accurate and stable estimates than existing alternatives. Additionally, the generated instability paths offer a valuable graphical tool for understanding the inlier and outlier distributions within the data. The method naturally extends to generalized linear models, where we find that variance-stabilizing transformations produce residuals that are well-suited for outlier detection. Applications to two real-world datasets further illustrate the practical utility of our approach in identifying outliers.

statistics & probability,computer science, theory & methods
The Limit Distribution of the Bootstrap for the Unit Root Test Statistic when the Residuals Are Dependent

Li-Xin Zhang,Xiao-Rong Yang

DOI: https://doi.org/10.1007/s00184-006-0070-y

IF: 0.96

2006-01-01

Metrika

Abstract:In this article, the unit root test for the AR(1) model with dependent residuals is considered. We adopt a bootstrap procedure to bootstrap the residuals with bootstrap sample size m less than the size n of the original sample. Under the assumptions that m → ∞ and m/n → 0, the convergence in probability of the bootstrap distribution function is established.
The Bootstrap for Dynamical Systems

Kasun Fernando,Nan Zou

DOI: https://doi.org/10.48550/arXiv.2108.08461

2021-08-19

Abstract:Despite their deterministic nature, dynamical systems often exhibit seemingly random behaviour. Consequently, a dynamical system is usually represented by a probabilistic model of which the unknown parameters must be estimated using statistical methods. When measuring the uncertainty of such parameter estimation, the bootstrap stands out as a simple but powerful technique. In this paper, we develop the bootstrap for dynamical systems and establish not only its consistency but also its second-order efficiency via a novel \textit{continuous} Edgeworth expansion for dynamical systems. This is the first time such continuous Edgeworth expansions have been studied. Moreover, we verify the theoretical results about the bootstrap using computer simulations.

Dynamical Systems,Statistics Theory
Consistency of the bootstrap for asymptotically linear estimators based on machine learning

Zhou Tang,Ted Westling

2024-04-18

Abstract:The bootstrap is a popular method of constructing confidence intervals due to its ease of use and broad applicability. Theoretical properties of bootstrap procedures have been established in a variety of settings. However, there is limited theoretical research on the use of the bootstrap in the context of estimation of a differentiable functional in a nonparametric or semiparametric model when nuisance functions are estimated using machine learning. In this article, we provide general conditions for consistency of the bootstrap in such scenarios. Our results cover a range of estimator constructions, nuisance estimation methods, bootstrap sampling distributions, and bootstrap confidence interval types. We provide refined results for the empirical bootstrap and smoothed bootstraps, and for one-step estimators, plug-in estimators, empirical mean plug-in estimators, and estimating equations-based estimators. We illustrate the use of our general results by demonstrating the asymptotic validity of bootstrap confidence intervals for the average density value and G-computed conditional mean parameters, and compare their performance in finite samples using numerical studies. Throughout, we emphasize whether and how the bootstrap can produce asymptotically valid confidence intervals when standard methods fail to do so.

Statistics Theory
Software for Data-Based Stochastic Programming Using Bootstrap Estimation

Xiaotie Chen,David L. Woodruff

DOI: https://doi.org/10.1287/ijoc.2022.0253

IF: 3.288

2023-07-25

INFORMS Journal on Computing

Abstract:We describe software for stochastic programming that uses only sampled data to obtain both a consistent sample-average solution and a consistent estimate of confidence intervals for the optimality gap using bootstrap and bagging. The underlying distribution whence the samples come is not required. History: Accepted by Ted Ralphs, Area Editor for Software Tools. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0253 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0253 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

computer science, interdisciplinary applications,operations research & management science
Bootstrapping Financial Time Series

Esther Ruiz,Lorenzo Pascual

DOI: https://doi.org/10.1111/1467-6419.00170

2002-07-01

Journal of Economic Surveys

Abstract:It is well known that time series of returns are characterized by volatility clustering and excess kurtosis. Therefore, when modelling the dynamic behavior of returns, inference and prediction methods, based on independent and/or Gaussian observations may be inadequate. As bootstrap methods are not, in general, based on any particular assumption on the distribution of the data, they are well suited for the analysis of returns. This paper reviews the application of bootstrap procedures for inference and prediction of financial time series. In relation to inference, bootstrap techniques have been applied to obtain the sample distribution of statistics for testing, for example, autoregressive dynamics in the conditional mean and variance, unit roots in the mean, fractional integration in volatility and the predictive ability of technical trading rules. On the other hand, bootstrap procedures have been used to estimate the distribution of returns which is of interest, for example, for Value at Risk (VaR) models or for prediction purposes. Although the application of bootstrap techniques to the empirical analysis of financial time series is very broad, there are few analytical results on the statistical properties of these techniques when applied to heteroscedastic time series. Furthermore, there are quite a few papers where the bootstrap procedures used are not adequate.

economics
Bootstrap method based evaluating VaR

叶五一,缪柏其,吴振翔

DOI: https://doi.org/10.3969/j.issn.1000-5781.2004.05.016

2004-01-01

Abstract:In this paper the nonparametic Bootstrap method is introduced and the VaR of the financial capital is evaluated. Because the financial asset yield is not easy to be estimated by parametic method, as a nonparametic method, Bootstrap method overcomes those defects, and also improves the history simulation method. In this paper VaR of European dollar and Japanese yen exchange rates are calculated, both point estimator and interval estimator are given. By comparing those methods, some significative results are found.
Bootstrap methods for stationary functional time series

Han Lin Shang

DOI: https://doi.org/10.48550/arXiv.1610.00773

2016-10-04

Abstract:Bootstrap methods for estimating the long-run covariance of stationary functional time series are considered. We introduce a versatile bootstrap method that relies on functional principal component analysis, where principal component scores can be bootstrapped by maximum entropy. Two other bootstrap methods resample error functions, after the dependence structure being modeled linearly by a sieve method or nonlinearly by a functional kernel regression. Through a series of Monte-Carlo simulation, we evaluate and compare the finite-sample performances of these three bootstrap methods for estimating the long-run covariance in a functional time series. Using the intraday particulate matter (PM10) data set in Graz, the proposed bootstrap methods provide a way of constructing the distribution of estimated long-run covariance for functional time series.

Computation
On the failure of the bootstrap for Chatterjee's rank correlation

Zhexiao Lin,Fang Han

DOI: https://doi.org/10.1093/biomet/asae004

IF: 3.0279

2024-02-04

Biometrika

Abstract:Summary While researchers commonly use the bootstrap to quantify the uncertainty of an estimator, it has been noticed that the standard bootstrap, in general, does not work for Chatterjee's rank correlation. In this paper, we provide proof of this issue under an additional independence assumption, and complement our theory with simulation evidence for general settings. Chatterjee's rank correlation thus falls into a category of statistics that are asymptotically normal but bootstrap inconsistent. Valid inferential methods in this case are Chatterjee's original proposal for testing independence and Lin & Han (2022) 's analytic asymptotic variance estimator for more general purposes.

statistics & probability,mathematical & computational biology,biology
Model-free Bootstrap for a General Class of Stationary Time Series

Yiren Wang,Dimitris N. Politis

DOI: https://doi.org/10.48550/arXiv.1912.13185

2019-12-31

Abstract:A model-free bootstrap procedure for a general class of stationary time series is introduced. The theoretical framework is established, showing asymptotic validity of bootstrap confidence intervals for many statistics of interest. In addition, asymptotic validity of one-step ahead bootstrap prediction intervals is also demonstrated. Finite-sample experiments are conducted to empirically confirm the performance of the new method, and to compare with popular methods such as the block bootstrap and the autoregressive (AR)-sieve bootstrap.

Statistics Theory
The Big Data Bootstrap

Ariel Kleiner,Ameet Talwalkar,Purnamrita Sarkar,Michael Jordan

DOI: https://doi.org/10.48550/arXiv.1206.6415

2012-06-28

Abstract:The bootstrap provides a simple and powerful means of assessing the quality of estimators. However, in settings involving large datasets, the computation of bootstrap-based quantities can be prohibitively demanding. As an alternative, we present the Bag of Little Bootstraps (BLB), a new procedure which incorporates features of both the bootstrap and subsampling to obtain a robust, computationally efficient means of assessing estimator quality. BLB is well suited to modern parallel and distributed computing architectures and retains the generic applicability, statistical efficiency, and favorable theoretical properties of the bootstrap. We provide the results of an extensive empirical and theoretical investigation of BLB's behavior, including a study of its statistical correctness, its large-scale implementation and performance, selection of hyperparameters, and performance on real data.

Machine Learning
On the relative performance of the block bootstrap for dependent data

Bing-Yi Jing

DOI: https://doi.org/10.1080/03610929708831984

1997-01-01

Abstract:In the independent setting, both Efron's bootstrap and “empiricai Edgeworth expansion” (E.E-expansion) give second-order accurate approximations to distributions of standardized and studentized statistics in the smooth function model. As a result, Efron's bootstrap was often regarded as roughly equivalent to the one-term E.E-expansion. However, a more detailed analysis shows that Efron's bootstrap outperforms the E.E-expansion in terms of loss functions by Bhattacharya and Qumsiyeh (1989) and in terms of probabilities for large deviations by Hall (1990) and Jing et a1 (1994). in this paper, we shall study the performances of the block bootstrap and the E.E-expansion for the weakly dependent data. It turns out that similar properties hold:both perform equally well at the center of the distribution but the block bootstrap provides accurate approximations even in the tails of the distributions. The study is focued on the simple case of standardized and studentized sample mean, but the conclusions can be easily extended to the smooth function of multivariate means.

Estimating the variability of the Stein estimator by bootstrap

Cluster-Robust Bootstrap Inference in Quantile Regression Models

Wild bootstrap inference for penalized quantile regression for longitudinal data

Data Sharpening Method in Regression Confidence Band

Bootstrap Inference for Panel Data Quantile Regression

The Grid Bootstrap for Continuous Time Models

Bootstrap SGD: Algorithmic Stability and Robustness

Bootstrapping non-stationary stochastic volatility

Bootstrapping the Cross-Validation Estimate

Bootstrap estimation of the proportion of outliers in robust regression

The Limit Distribution of the Bootstrap for the Unit Root Test Statistic when the Residuals Are Dependent

The Bootstrap for Dynamical Systems

Consistency of the bootstrap for asymptotically linear estimators based on machine learning

Software for Data-Based Stochastic Programming Using Bootstrap Estimation

Bootstrapping Financial Time Series

Bootstrap method based evaluating VaR

Bootstrap methods for stationary functional time series

On the failure of the bootstrap for Chatterjee's rank correlation

Model-free Bootstrap for a General Class of Stationary Time Series

The Big Data Bootstrap

On the relative performance of the block bootstrap for dependent data