Abstract:In benchmarking theory, creating a comprehensive and uniformly distributed set of problems is a crucial first step to designing a good benchmark. However, this step is also one of the hardest, as it can be difficult to determine how to evaluate the quality of the chosen problem set.In this article, we evaluate if the field of exploratory landscape analysis can be used to develop a generalized method of visualizing a set of arbitrary optimization functions. We present a method for visually determining the distribution of problems within a benchmark set using exploratory landscape analysis combined with clustering and t-sne visualization, and evaluate and explain the visualization this methodology produces.The proposed method is evaluated on a set of benchmark problems taken from two well known state-of-the-art real-parameter single objective optimization benchmarks: the CEC Special Sessions and Competitions on Real-Parameter Single Objective optimization, and the GECCO Black-Box Optimization Benchmark workshops.The main goal of this paper is to present an analysis of how exploratory landscape analysis can be used to visualize a benchmark problem set. We show that this method can provide a clear visualization of a benchmark problem set and shows the similarities of the problems in it by placing similar problems visually close together. We also show that the problem sets of the above benchmarks have a somewhat distinct set of problems that do not overlap.In addition, by applying feature selection approaches we show that a number of landscape features provided by state-of-the-art exploratory landscape analysis libraries are redundant and that a large amount of them are not invariant to simple transforms like scaling and shifting, at least when analyzing these two datasets.

Decomposition-Based Multi-objective Landscape Features and Automated Algorithm Selection.

Exploratory Landscape Analysis for Mixed-Variable Problems

A Collection of Deep Learning-based Feature-Free Approaches for Characterizing Single-Objective Continuous Fitness Landscapes

Multiple landscape measure-based approach for dynamic optimization problems

Decomposition Multi-Objective Evolutionary Optimization: From State-of-the-Art to Future Opportunities

A Landscape-Aware Differential Evolution for Multimodal Optimization Problems

A survey of decomposition approaches in multiobjective evolutionary algorithms

Landscape-Aware Automated Algorithm Configuration using Multi-output Mixed Regression and Classification

Understanding the problem space in single-objective numerical optimization using exploratory landscape analysis

Decomposition Multi-Objective Evolutionary Algorithm Based on Adaptive Neighborhood Adjustment Strategy

Analyzing and Overcoming Local Optima in Complex Multi-Objective Optimization by Decomposition-Based Evolutionary Algorithms

A Multi-Task Decomposition-Based Evolutionary Algorithm for Tackling High-Dimensional Bi-Objective Feature Selection

A Decomposition-based Large-scale Multi-modal Multi-objective optimization Algorithm

Set-based Multiobjective Fitness Landscapes: A Preliminary Study

Decomposition in decision and objective space for multi-modal multi-objective optimization

A Survey of Decomposition-Based Evolutionary Multi-Objective Optimization: Part I-Past and Future

A Decomposition Based Evolutionary Algorithm with Angle Penalty Selection Strategy for Many-Objective Optimization.

A localized decomposition evolutionary algorithm for imbalanced multi-objective optimization

Pareto Local Optima of Multiobjective NK-Landscapes with Correlated Objectives

A General-Purpose Tunable Landscape Generator

Landscape-Sketch-Step: An AI/ML-Based Metaheuristic for Surrogate Optimization Problems