Abstract:Discrete-event simulation models generate random variates from input distributions and compute outputs according to the simulation logic. The input distributions are typically fitted to finite real-world data and thus are subject to estimation errors that can propagate to the simulation outputs: an issue commonly known as input uncertainty (IU). This paper investigates quantifying IU using the output confidence intervals (CIs) computed from bootstrap quantile estimators. The standard direct bootstrap method has overcoverage due to convolution of the simulation error and IU; however, the brute-force way of washing away the former is computationally demanding. We present two new bootstrap methods to enhance direct resampling in both statistical and computational efficiencies using shrinkage strategies to down-scale the variabilities encapsulated in the CIs. Our asymptotic analysis shows how both approaches produce tight CIs accounting for IU under limited input data and simulation effort along with the simulation sample-size requirements relative to the input data size. We demonstrate performances of the shrinkage strategies with several numerical experiments and investigate the conditions under which each method performs well. We also show advantages of nonparametric approaches over parametric bootstrap when the distribution family is misspecified and over metamodel approaches when the dimension of the distribution parameters is high. History: Accepted by Bruno Tuffin, Area Editor for Simulation. Funding: This work was supported by the National Science Foundation [CAREER CMMI-1834710, CAREER CMMI-2045400, DMS-1854659, and IIS-1849280]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0044 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0044 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Confidence Intervals from Stochastic Approximation

A Bayesian Stochastic Approximation Method

Simultaneous Confidence Intervals from Randomization Tests with Application in Testing Bioequivalence with Multiple Endpoints.

Robust Confidence Bands for Stochastic Processes Using Simulation

Adaptive Designs for Stochastic Root-Finding

Formalization of a Stochastic Approximation Theorem

High Confidence Level Inference is Almost Free using Parallel Stochastic Optimization

On randomized confidence intervals for the binomial probability

Estimation and Inference by Stochastic Optimization: Three Examples

Robust Stochastic Approximation Approach to Stochastic Programming

Computation of VaR and CVaR using stochastic approximations and unconstrained importance sampling

A Self-Normalized Approach to Confidence Interval Construction in Time Series

Scalable Statistical Inference in Non-parametric Least Squares

Guaranteed Conservative Fixed Width Confidence Intervals Via Monte Carlo Sampling

Overlapping Batch Confidence Intervals on Statistical Functionals Constructed from Time Series: Application to Quantiles, Optimization, and Estimation

Confidence intervals in monotone regression

Application of Bootstrap Estimators in Simulation Experiment

A Shrinkage Approach to Improve Direct Bootstrap Resampling Under Input Uncertainty

Stochastic Approximation for Expectation Objective and Expectation Inequality-Constrained Nonconvex Optimization

A Stochastic Approximation Framework for a Class of Randomized Optimization Algorithms