Abstract:We study the construction of a confidence interval (CI) for a simulation output performance measure that accounts for input uncertainty when the input models are estimated from finite data. In particular, we focus on performance measures that can be expressed as a ratio of two dependent simulation outputs' means. We adopt the parametric bootstrap method to mimic input data sampling and construct the percentile bootstrap CI after estimating the ratio at each bootstrap sample. The standard estimator, which takes the ratio of two sample averages, tends to exhibit large finite-sample bias and variance, leading to overcoverage of the percentile bootstrap CI. To address this, we propose two new ratio estimators that replace the sample averages with pooled mean estimators via the $k$-nearest neighbor ($k$NN) regression: the $k$NN estimator and the $k$LR estimator. The $k$NN estimator performs well in low dimensions but its theoretical performance guarantee degrades as the dimension increases. The $k$LR estimator combines the likelihood ratio (LR) method with the $k$NN regression, leveraging the strengths of both while mitigating their weaknesses; the LR method removes dependence on dimension, while the variance inflation introduced by the LR is controlled by $k$NN. Based on asymptotic analyses and finite-sample heuristics, we propose an experiment design that maximizes the efficiency of the proposed estimators and demonstrate their empirical performances using three examples including one in the enterprise risk management application.

Robust Simulation with Likelihood-Ratio Constrained Input Uncertainty

Robust Simulation of Stochastic Systems with Input Uncertainties Modeled by Statistical Divergences

Distributionally Robust Selection of the Best

Multistage Robust Mixed-Integer Optimization under Endogenous Uncertainty

Simulation-Based Robust Design of Complex Product Considering Uncertainties of Metamodel, Design Variables, and Noise Parameters

Distributionally Robust Stratified Sampling for Stochastic Simulations with Multiple Uncertain Input Models

Upper-Confidence-Bound Procedure for Robust Selection of The Best.

New Robust Optimization Approach Induced by Flexible Uncertainty Set: Optimization under Continuous Uncertainty

A Nonparametric Bayesian Framework for Uncertainty Quantification in Stochastic Simulation

Robust Stochastic Optimization Made Easy with RSOME

Risk Quantification in Stochastic Simulation under Input Uncertainty

Unified Reliability Measure Method Considering Uncertainties of Input Variables and Their Distribution Parameters

Data-driven Ranking and Selection under Input Uncertainty

Robust Confidence Bands for Stochastic Processes Using Simulation

Efficient Input Uncertainty Quantification for Ratio Estimator

Quantifying Distributional Input Uncertainty via Inflated Kolmogorov-Smirnov Confidence Band

Statistical Uncertainty Analysis for Stochastic Simulation

Heterogeneous uncertainty quantification using Bayesian inference for simulation-based design optimization

METAMODEL-ASSISTED RISK ANALYSIS FOR STOCHASTIC SIMULATION WITH INPUT UNCERTAINTY

Likelihood robust optimization for data-driven problems

On the risk levels of distributionally robust chance constrained problems