Abstract:Heuristic optimisation algorithms explore the search space by sampling solutions, evaluating their fitness, and biasing the search in the direction of promising solutions. However, in many cases, this fitness function involves executing expensive computational calculations, drastically reducing the reasonable number of evaluations. In this context, surrogate models have emerged as an excellent alternative to alleviate these computational problems. This paper addresses the formulation of surrogate problems as both regression models that approximate fitness (surface surrogate models) and a novel way to connect classification models (pairwise surrogate models). The pairwise approach can be directly exploited by some algorithms, such as Differential Evolution, in which the fitness value is not actually needed to drive the search, and it is sufficient to know whether a solution is better than another one or not. Based on these modelling approaches, we have conducted a multidimensional analysis of surrogate models under different configurations: different machine learning algorithms (regularised regression, neural networks, decision trees, boosting methods, and random forests), different surrogate strategies (encouraging diversity or relaxing prediction thresholds), and compare them for both surface and pairwise surrogate models. The experimental part of the article includes the benchmark problems already proposed for the SOCO2011 competition in continuous optimisation and a simulation problem included in the recent GECCO2021 Industrial Challenge. This paper shows that the performance of the overall search, when using online machine learning-based surrogate models, depends not only on the accuracy of the predictive model but also on both the kind of bias towards positive or negative cases and how the optimisation uses those predictions to decide whether to execute the actual fitness function.

Landscape Analysis for Surrogate Models in the Evolutionary Black-Box Context

Temporal True and Surrogate Fitness Landscape Analysis for Expensive Bi-Objective Optimisation

Surrogate-assisted Bayesian inversion for landscape and basin evolution models

Comparative study of regression vs pairwise models for surrogate-based heuristic optimisation

Understanding the problem space in single-objective numerical optimization using exploratory landscape analysis

Explainable Landscape Analysis in Automated Algorithm Performance Prediction

Exploratory Landscape Analysis for Mixed-Variable Problems

Neural Exploratory Landscape Analysis

Modulating Surrogates for Bayesian Optimization

Exploratory Landscape Validation for Bayesian Optimization Algorithms

Two Gaussian Approaches to Black-Box Optomization

Landscape-Sketch-Step: An AI/ML-Based Metaheuristic for Surrogate Optimization Problems

A Surrogate-Assisted Evolutionary Algorithm with Random Feature Selection for Large-Scale Expensive Problems

Advances in surrogate based modeling, feasibility analysis, and optimization: A review

A Novel and Fully Automated Domain Transformation Scheme for Near Optimal Surrogate Construction

Model Uncertainty in Evolutionary Optimization and Bayesian Optimization: A Comparative Analysis

Combining Lipschitz and RBF Surrogate Models for High-dimensional Computationally Expensive Problems

Linear Subspace Surrogate Modeling for Large-Scale Expensive Single/Multi-Objective Optimization

Ensemble of surrogates in black-box-type engineering optimization: Recent advances and applications

A Collection of Deep Learning-based Feature-Free Approaches for Characterizing Single-Objective Continuous Fitness Landscapes

Expected Improvement versus Predicted Value in Surrogate-Based Optimization