An Acceptable Design versus an Exactly Identified Top Group
Abstract
Good-enough and top-design selection allocate error differently. Their efficiency comparison should hold operational requirements common or explicitly report the task difference. Tolerance sensitivity and independent membership checks reveal when fine ranking has value. A procedure should not receive credit for solving a relaxed adequacy task while being described as precise top-group identification. This focused synthesis develops an empirical comparison that preserves the terminal action, observation process, and total verification cost. Its proposed tests are prospective; no new simulation run, model-training result, or intervention effect is reported.
References
F. Gao and S. Gao, “A New Strategy for Selecting Good Enough Designs Using Optimal Computing Budget Allocation,” Journal of Simulation, vol. 12, no. 3, pp. 238–247, 2018.
S. Gao and W. Chen, “A New Budget Allocation Framework for Selecting Top Simulated Designs,” IIE Transactions, vol. 48, no. 9, pp. 855–863, 2016.
Sacks, Jerome, William J. Welch, Toby J. Mitchell, and Henry P. Wynn. “Design and Analysis of Computer Experiments.” Statistical Science, vol. 4, no. 4, 1989, pp. 409–423. https://doi.org/10.1214/ss/1177012413.
Bertsimas, Dimitris, and Melvyn Sim. “The Price of Robustness.” Operations Research, vol. 52, no. 1, 2004, pp. 35–53. https://doi.org/10.1287/opre.1030.0065.
Campi, Marco C., and Simone Garatti. “The Exact Feasibility of Randomized Solutions of Uncertain Convex Programs.” SIAM Journal on Optimization, vol. 19, no. 3, 2008, pp. 1211–1230. https://doi.org/10.1137/07069821X.
Glasserman, Paul. Monte Carlo Methods in Financial Engineering. Springer, 2004. https://doi.org/10.1007/978-0-387-21617-1.
Ankenman, Bruce, Barry L. Nelson, and Jeremy Staum. “Stochastic Kriging for Simulation Metamodeling.” Operations Research, vol. 58, no. 2, 2010, pp. 371–382. https://doi.org/10.1287/opre.1090.0754.
Kim, Seong-Hee, and Barry L. Nelson. “A Fully Sequential Procedure for Indifference-Zone Selection in Simulation.” ACM Transactions on Modeling and Computer Simulation, vol. 11, no. 3, 2001, pp. 251–273. https://doi.org/10.1145/502109.502111.
Chick, Stephen E., and Koichiro Inoue. “New Two-Stage and Sequential Procedures for Selecting the Best Simulated System.” Operations Research, vol. 49, no. 5, 2001, pp. 732–743. https://doi.org/10.1287/opre.49.5.732.10615.
Frazier, Peter I., Warren B. Powell, and Savas Dayanik. “A Knowledge-Gradient Policy for Sequential Information Collection.” SIAM Journal on Control and Optimization, vol. 47, no. 5, 2008, pp. 2410–2439. https://doi.org/10.1137/070693424.