Abstract:In recent years, with the rapid development of computing technology, developing parallel procedures to solve large-scale ranking and selection (R&S) problems has attracted a lot of research attention. In this paper, we take fixed-budget R&S procedure as an example to investigate potential issues of developing parallel procedures. We argue that to measure the performance of a fixed-budget R&S procedure in solving large-scale problems, it is important to quantify the minimal growth rate of the total sampling budget such that as the number of alternatives increases, the probability of correct selection (PCS) would not decrease to zero. We call such a growth rate of the total sampling budget the rate for maintaining correct selection (RMCS). We show that a tight lower bound for the RMCS of a broad class of existing fixed-budget procedures is in the order of [Formula: see text], where k is the number of alternatives. Then, we propose a new type of fixed-budget procedure, namely the fixed-budget knockout-tournament ([Formula: see text]) procedure. We prove that, in terms of the RMCS, our procedure outperforms existing fixed-budget procedures and achieves the optimal order, that is, the order of k. Moreover, we demonstrate that our procedure can be easily implemented in parallel computing environments with almost no nonparallelizable calculations. Last, a comprehensive numerical study shows that our procedure is indeed suitable for solving large-scale problems in parallel computing environments. History: Accepted by Bruno Tuffin, Area Editor for Simulation. Funding: Y. Zhong was supported by the National Natural Science Foundation of China [Grant 72101047]. L. J. Hong was supported by the National Natural Science Foundation of China [Grants 72091211 and 72161160340]. G. Jiang was supported by the National Natural Science Foundation of China [Grants 72121001 and 72171060]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/ijoc.2022.1221 .

A rank method for testing perfect ranking in multi-cycle ranked set sampling

Permutation-Based Tests of Perfect Ranking

A Rank-Based Sampling Framework for Offline Reinforcement Learning

Investigating the Impact of Ranked Set Sampling in Nonparametric Cusum Control Charts

Birnbaum-Saunders parameters estimation using simple random sampling and ranked set sampling

APPROXIMATION TO THE DISTRIBUTION OF M-ESTIMATES IN RANKED-SET SAMPLING BY RANDOMLY WEIGHTED BOOTSTRAP

Power comparison of the Kolmogorov–Smirnov test under ranked set sampling and simple random sampling

Extropy properties of ranked set sample when ranking is not perfect

Algorithm for Calculating the Initial Sample Size in a Fully Sequential Ranking and Selection Procedure

Ranking Inferences Based on the Top Choice of Multiway Comparisons

Precedence Tests for Equality of Two Distributions Based on Early Failures of Ranked Set Samples

Global Rank Sum Test: An Efficient Rank-Based Nonparametric Test for Large Scale Online Experiment

Optimal Computing Budget Allocation for Data-driven Ranking and Selection

Asymptotically Optimal Sequential Design for Rank Aggregation

Sequential sampling for a ranking and selection problem with exponential sampling distributions

An Indifference-Zone Selection Procedure with Minimum Switching and Sequential Sampling

A REVIEW OF STATIC AND DYNAMIC OPTIMIZATION FOR RANKING AND SELECTION

SPECTRAL METHOD AND REGULARIZED MLE ARE BOTH OPTIMAL FOR TOP- K RANKING

Solving Large-Scale Fixed-Budget Ranking and Selection Problems

A New Probabilistic Model for Rank Aggregation.

A Count-Based Nonparametric Test on Strict Bivariate Stochastic Arrangement Increasing Property