Abstract:In benchmarking theory, creating a comprehensive and uniformly distributed set of problems is a crucial first step to designing a good benchmark. However, this step is also one of the hardest, as it can be difficult to determine how to evaluate the quality of the chosen problem set.In this article, we evaluate if the field of exploratory landscape analysis can be used to develop a generalized method of visualizing a set of arbitrary optimization functions. We present a method for visually determining the distribution of problems within a benchmark set using exploratory landscape analysis combined with clustering and t-sne visualization, and evaluate and explain the visualization this methodology produces.The proposed method is evaluated on a set of benchmark problems taken from two well known state-of-the-art real-parameter single objective optimization benchmarks: the CEC Special Sessions and Competitions on Real-Parameter Single Objective optimization, and the GECCO Black-Box Optimization Benchmark workshops.The main goal of this paper is to present an analysis of how exploratory landscape analysis can be used to visualize a benchmark problem set. We show that this method can provide a clear visualization of a benchmark problem set and shows the similarities of the problems in it by placing similar problems visually close together. We also show that the problem sets of the above benchmarks have a somewhat distinct set of problems that do not overlap.In addition, by applying feature selection approaches we show that a number of landscape features provided by state-of-the-art exploratory landscape analysis libraries are redundant and that a large amount of them are not invariant to simple transforms like scaling and shifting, at least when analyzing these two datasets.

Spatial-domain fitness landscape analysis for combinatorial optimization

A novel stochastic method with modified extremal optimization and nearest neighbor search for hard combinatorial problems

A survey of fitness landscape analysis for optimization

Visual analysis of fitness landscapes in architectural design optimization

A Multi-Objective Optimization of Physical Activity Spaces

A New Fitness-Landscape-Driven Particle Swarm Optimization

A Hybrid Method of Combinatorial Search and Coordinate Descent for Discrete Optimization

Local Optima Networks: A New Model of Combinatorial Fitness Landscapes

Set-based Multiobjective Fitness Landscapes: A Preliminary Study

A probabilistic framework with the gradient-based method for multi-objective land use optimization

Fitness Landscape Footprint: A Framework to Compare Neural Architecture Search Problems

A Collection of Deep Learning-based Feature-Free Approaches for Characterizing Single-Objective Continuous Fitness Landscapes

Understanding the problem space in single-objective numerical optimization using exploratory landscape analysis

New Methods of Studying Valley Fitness Landscapes

Visually-driven parallel solving of multi-objective land-use allocation problems: a case study in Chelan, Washington

Pareto Local Optima of Multiobjective NK-Landscapes with Correlated Objectives

A General-Purpose Tunable Landscape Generator

Competitive model for combinatorial optimization problems

A Geometric Structure-Based Particle Swarm Optimization Algorithm for Multiobjective Problems

A distance and cosine similarity-based fitness evaluation mechanism for large-scale many-objective optimization

Local Optima Networks, Landscape Autocorrelation and Heuristic Search Performance