Abstract:Differential evolution (DE) is a simple and effective approach for solving numerical optimization problems. However, the performance of DE is sensitive to the choice of mutation and crossover strategies and their associated control parameters. Therefore, to achieve optimal performance, a time-consuming parameter tuning process is required. In DE, the use of different mutation and crossover strategies with different parameter settings can be appropriate during different stages of the evolution. Therefore, to achieve optimal performance using DE, various adaptation, self-adaptation, and ensemble techniques have been proposed. Recently, a classification-assisted DE algorithm was proposed to overcome trial and error parameter tuning and efficiently solve computationally expensive problems. In this paper, we present an evolving surrogate model-based differential evolution (ESMDE) method, wherein a surrogate model constructed based on the population members of the current generation is used to assist the DE algorithm in order to generate competitive offspring using the appropriate parameter setting during different stages of the evolution. As the population evolves over generations, the surrogate model also evolves over the iterations and better represents the basin of search by the DE algorithm. The proposed method employs a simple Kriging model to construct the surrogate. The performance of ESMDE is evaluated on a set of 17 bound-constrained problems. The performance of the proposed algorithm is compared to state-of-the-art self-adaptive DE algorithms: the classification-assisted DE algorithm, regression-assisted DE algorithm, and ranking-assisted DE algorithm.

Self-guided Evolution Strategies with Historical Estimated Gradients

Improving Gradient Estimation in Evolutionary Strategies With Past Descent Directions

Gradient Q : A Unified Algorithm with Function Approximation for Reinforcement Learning

Guided evolutionary strategies: Augmenting random search with surrogate gradients

Accelerating Reinforcement Learning with a Directional-Gaussian-Smoothing Evolution Strategy

Gradient Q(σ, Λ): A Unified Algorithm with Function Approximation for Reinforcement Learning

Maximum Entropy Reinforcement Learning with Evolution Strategies

Low-Variance Gradient Estimation in Unrolled Computation Graphs with ES-Single

Safe Reinforcement Learning Using Finite-Horizon Gradient-based Estimation

Stein Variational Evolution Strategies

Trust Region Evolution Strategies.

Instance Weighted Incremental Evolution Strategies for Reinforcement Learning in Dynamic Environments

Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents

Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategies

Large-Scale Evolution Strategy Based on Search Direction Adaptation

Proximal evolutionary strategy: improving deep reinforcement learning through evolutionary policy optimization

Challenges in High-Dimensional Reinforcement Learning with Evolution Strategies

Evolutionary Gradient Descent for Non-convex Optimization

Evolutionary Sampling Agent for Expensive Problems

Quality with Just Enough Diversity in Evolutionary Policy Search

An evolving surrogate model-based differential evolution algorithm