Non-monotonicity of Probability of Correct Selection

Yijie Peng,Chun-Hung Chen,Michael C. Fu,Jian-Qiang Hu
DOI: https://doi.org/10.1109/wsc.2015.7408526
2015-01-01
Abstract:In Peng et al. (2015b), we show that the probability of correct selection (PCS), a commonly used metric, is not necessarily monotonically increasing with respect to the number of simulation replications. A simple counterexample where the PCS may decrease with additional sampling is provided to motivate the problem. The reference identifies the induced correlations as the source of the non-monotonicity, and characterizes the general scenario under which the phenomenon occurs by a condition where coefficient of variations of the difference in sample means are large. Numerical examples further illustrate the non-monotonic behavior of the PCS for some well-known sampling schemes.
What problem does this paper attempt to address?