Abstract:Dynamic environments pose great challenges for expensive optimization problems, as the objective functions of these problems change over time and thus require remarkable computational resources to track the optimal solutions. Although data-driven evolutionary optimization and Bayesian optimization (BO) approaches have shown promise in solving expensive optimization problems in static environments, the attempts to develop such approaches in dynamic environments remain rarely unexplored. In this paper, we propose a simple yet effective meta-learning-based optimization framework for solving expensive dynamic optimization problems. This framework is flexible, allowing any off-the-shelf continuously differentiable surrogate model to be used in a plug-in manner, either in data-driven evolutionary optimization or BO approaches. In particular, the framework consists of two unique components: 1) the meta-learning component, in which a gradient-based meta-learning approach is adopted to learn experience (effective model parameters) across different dynamics along the optimization process. 2) the adaptation component, where the learned experience (model parameters) is used as the initial parameters for fast adaptation in the dynamic environment based on few shot samples. By doing so, the optimization process is able to quickly initiate the search in a new environment within a strictly restricted computational budget. Experiments demonstrate the effectiveness of the proposed algorithm framework compared to several state-of-the-art algorithms on common benchmark test problems under different dynamic characteristics.

What problem does this paper attempt to address?

The paper attempts to address the problem of efficiently solving Expensive Dynamic Optimization Problems (DOPs) in dynamic environments. Specifically, these problems are characterized by objective functions or constraints that change over time, and each evaluation of these functions requires significant computational resources. Therefore, when the environment changes, traditional optimization methods often struggle to quickly adapt to the new optimal solution, especially when computational resources are limited. The paper proposes an optimization framework based on meta-learning, aiming to accelerate the optimization process in new environments by leveraging knowledge obtained from previous tasks and improving the ability to find the optimal solution within a limited computational budget. ### Analysis of Core Issues in the Paper 1. **Challenges Brought by Dynamic Environments**: Due to the changing objective functions over time, it is necessary to continuously track the position of the optimal solution during the optimization process, which is particularly difficult in dynamic environments. 2. **Data Scarcity**: In expensive optimization problems, the high cost of each function evaluation results in a very limited amount of available data, further increasing the difficulty of optimization. 3. **Limitations of Existing Methods**: Although data-driven evolutionary optimization and Bayesian Optimization (BO) perform well in static environments, they are less frequently applied in dynamic environments. Most methods assume that early collected data is less useful than recent data, which may lead to the loss of valuable information. ### Overview of the Solution The paper proposes a meta-learning-based optimization framework (MLO), which includes two main components: 1. **Meta-Learning Component**: Using gradient-based meta-learning methods to learn effective initial model parameters from different dynamic environments, enabling a quick start of the new optimization process when the environment changes. 2. **Adaptation Component**: Utilizing the initial parameters obtained from meta-learning to quickly adapt in the new environment and optimize with a small number of samples. ### Main Contributions - **Effective Initialization in Dynamic Environments**: Through the meta-learning mechanism, generating effective initial parameters when the environment changes, thus quickly starting the optimization process in the new environment. - **Efficient Learning with Limited Data**: Designing a framework suitable for few-shot learning scenarios, capable of effectively extracting domain-specific features with limited data. - **High Flexibility**: The framework can be combined with existing evolutionary optimization and Bayesian optimization methods, supporting any continuously differentiable surrogate model. - **Experimental Validation**: Demonstrating the effectiveness and superiority of the proposed algorithm through experiments on multiple benchmark problems, particularly in terms of solution quality, computational cost efficiency, and flexibility. In summary, by introducing meta-learning methods, the paper successfully addresses the key challenges of expensive optimization problems in dynamic environments, providing new ideas and tools for research in related fields.

Solving Expensive Optimization Problems in Dynamic Environments with Meta-learning

A Data-Driven Evolutionary Transfer Optimization for Expensive Problems in Dynamic Environments

Experience-Based Evolutionary Algorithms for Expensive Optimization

A reinforcement learning approach for dynamic multi-objective optimization

Incremental Data-driven Optimization of Complex Systems in Nonstationary Environments

Transfer Learning-Based Dynamic Multiobjective Optimization Algorithms

Multiple landscape measure-based approach for dynamic optimization problems

An Environment-Driven Hybrid Evolutionary Algorithm for Dynamic Multi-Objective Optimization Problems

Transfer Learning based Dynamic Multiobjective Optimization Algorithms

Distributed Evolution Strategies with Multi-Level Learning for Large-Scale Black-Box Optimization

Surrogate-Assisted Evolutionary Framework for Data-Driven Dynamic Optimization

An accelerate Prediction Strategy for Dynamic Multi-Objective Optimization

A Framework Based on Historical Evolution Learning for Dynamic Multiobjective Optimization

A New Learning Based Dynamic Multi-Objective Optimisation Evolutionary Algorithm

A data augmentation based Kriging-assisted reference vector guided evolutionary algorithm for expensive dynamic multi-objective optimization

Double Meta-Learning for Data Efficient Policy Optimization in Non-Stationary Environments

Optimization-Driven Adaptive Experimentation

Evolutionary Multi-Objective Bayesian Optimization Based on Multisource Online Transfer Learning

A Fast Dynamic Evolutionary Multiobjective Algorithm via Manifold Transfer Learning

An Adaptive Multi-Strategy Algorithm Based on Extent of Environmental Change for Dynamic Multiobjective Optimization

A reinforcement learning-based multi-objective optimization in an interval and dynamic environment