Abstract:Discrete-event simulation models generate random variates from input distributions and compute outputs according to the simulation logic. The input distributions are typically fitted to finite real-world data and thus are subject to estimation errors that can propagate to the simulation outputs: an issue commonly known as input uncertainty (IU). This paper investigates quantifying IU using the output confidence intervals (CIs) computed from bootstrap quantile estimators. The standard direct bootstrap method has overcoverage due to convolution of the simulation error and IU; however, the brute-force way of washing away the former is computationally demanding. We present two new bootstrap methods to enhance direct resampling in both statistical and computational efficiencies using shrinkage strategies to down-scale the variabilities encapsulated in the CIs. Our asymptotic analysis shows how both approaches produce tight CIs accounting for IU under limited input data and simulation effort along with the simulation sample-size requirements relative to the input data size. We demonstrate performances of the shrinkage strategies with several numerical experiments and investigate the conditions under which each method performs well. We also show advantages of nonparametric approaches over parametric bootstrap when the distribution family is misspecified and over metamodel approaches when the dimension of the distribution parameters is high. History: Accepted by Bruno Tuffin, Area Editor for Simulation. Funding: This work was supported by the National Science Foundation [CAREER CMMI-1834710, CAREER CMMI-2045400, DMS-1854659, and IIS-1849280]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0044 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0044 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Confidence Limits for the Indirect Effect: Distribution of the Product and Resampling Methods

Advantages of Monte Carlo Confidence Intervals for Indirect Effects

Estimating the prevalance of indirect effects and other spillovers

Correcting the Bias Correction for the Bootstrap Confidence Interval in Mediation Analysis

Empirical Likelihood Based Tests for Detecting the Presence of Significant Predictors in Marginal Quantile Regression

Confidence Intervals for the Probability of Superiority Effect Size Measure and the Area Under a Receiver Operating Characteristic Curve

Estimating the natural indirect effect and the mediation proportion via the product method

Equivalence of Two Least-Squares Estimators for Indirect Effects

Smaller $p$-values via indirect information

Conflating marginal and conditional treatment effects: Comments on 'Assessing the performance of population adjustment methods for anchored indirect comparisons: A simulation study'

Generic Inference on Quantile and Quantile Effect Functions for Discrete Outcomes

Indirect Effects in Sequential Mediation Models: Evaluating Methods for Hypothesis Testing and Confidence Interval Formation

Approximate confidence interval construction for risk difference under inverse sampling

A Shrinkage Approach to Improve Direct Bootstrap Resampling Under Input Uncertainty

A simulation study to compare methods for constructing confidence intervals for the incremental cost-effectiveness ratio

Enhanced inference for distributions and quantiles of individual treatment effects in various experiments

Robust inference on population indirect causal effects: the generalized front-door criterion

A Monte Carlo Study of Confidence Interval Methods for Generalizability Coefficient.

Causal effects based on distributional distances

Comparison of Software Algorithms for Calculating Reml Wald Type Confidence Limits for the Between-Group Variance Component in A Small Sample One-Way Random Effects Model Example

Estimating confidence regions of common measures of (baseline, treatment effect) on dichotomous outcome of a population