Abstract:This paper focuses on the performance evaluation of complex man-made systems, such as assembly lines, electric power grid, traffic systems, and various paper processing bureaucracies, etc. For such problems, applying the traditional optimization tool of mathematical programming and gradient descent procedures of continuous variables optimization are often inappropriate or infeasible, as the design variables are usually discrete and the accurate evaluation of the system performance via a simulation model can take too much calculation. General search type and heuristic methods are the only two methods to tackle the problems. However, the "goodness" of heuristic methods is generally difficult to quantify while search methods often involve extensive evaluation of systems at many design choices in a large search space using a simulation model resulting in an infeasible computation burden. The purpose of this paper is to address these difficulties simultaneously by extending the recently developed methodology of Ordinal Optimization (OO). Uniform samples are taken out from the whole search space and evaluated with a crude but computationally easy model when applying OO. And, we argue, after ordering via the crude performance estimates, that the lined-up uniform samples can be seen as an approximate ruler. By comparing the heuristic design with such a ruler, we can quantify the heuristic design, just as we measure the length of an object with a ruler. In a previous paper we showed how to quantify a heuristic design for a special case but we did not have the OO ruler idea at that time. In this paper we propose the OO ruler idea and extend the quantifying method to the general case and the multiple independent results case. Experimental results of applying the ruler are also given to illustrate the utility of this approach.

Implementation Issues in Optimization Algorithms: Do They Matter?

A novel stochastic method with modified extremal optimization and nearest neighbor search for hard combinatorial problems

Studies on Some Modern Optimization Algorithms

Two Criteria for Performance Analysis of Optimization Algorithms

Nature-inspired optimization algorithms: Challenges and open problems

No free lunch theorems for optimization

Two Possible Paradoxes In Numerical Comparisons Of Optimization Algorithms

A Novel Feature-Based Approach to Characterize Algorithm Performance for the Traveling Salesman Problem

Designing Adiabatic Quantum Optimization: A Case Study for the Traveling Salesman Problem

Optimization's Neglected Normative Commitments

Ordinal Optimization and Quantification of Heuristic Designs

Sufficient Conditions for Feasibility and Optimality of Real-Time Optimization Schemes - II. Implementation Issues

Paradoxes in Numerical Comparison of Optimization Algorithms

Learn to Optimize - A Brief Overview

Metaheuristic Optimization: Algorithm Analysis and Open Problems

Investigation of Bare-bones Algorithms from Quantum Perspective: A Quantum Dynamical Global Optimizer

Assessing the Effectiveness of (Parallel) Branch-and-bound Algorithms

A Novel Ranking Scheme for the Performance Analysis of Stochastic Optimization Algorithms using the Principles of Severity

Speedoo: prioritizing performance optimization opportunities.

Empirical Tests of Optimization Assumptions in Deep Learning

Average Case Subquadratic Exact and Heuristic Procedures for the Traveling Salesman 2-OPT Neighborhood