Abstract:Reinforcement learning (RL) is a machine learning approach that trains agents to maximize cumulative rewards through interactions with environments. The integration of RL with deep learning has recently resulted in impressive achievements in a wide range of challenging tasks, including board games, arcade games, and robot control. Despite these successes, there remain several crucial challenges, including brittle convergence properties caused by sensitive hyperparameters, difficulties in temporal credit assignment with long time horizons and sparse rewards, a lack of diverse exploration, especially in continuous search space scenarios, difficulties in credit assignment in multi-agent reinforcement learning, and conflicting objectives for rewards. Evolutionary computation (EC), which maintains a population of learning agents, has demonstrated promising performance in addressing these limitations. This article presents a comprehensive survey of state-of-the-art methods for integrating EC into RL, referred to as evolutionary reinforcement learning (EvoRL). We categorize EvoRL methods according to key research fields in RL, including hyperparameter optimization, policy search, exploration, reward shaping, meta-RL, and multi-objective RL. We then discuss future research directions in terms of efficient methods, benchmarks, and scalable platforms. This survey serves as a resource for researchers and practitioners interested in the field of EvoRL, highlighting the important challenges and opportunities for future research. With the help of this survey, researchers and practitioners can develop more efficient methods and tailored benchmarks for EvoRL, further advancing this promising cross-disciplinary research field.

Machine Learning in Evolutionary Economics

Evolutionary Learning: Advances in Theories and Algorithms

Evolutionary Multi-agent Reinforcement Learning in Group Social Dilemmas

Towards Theoretically Grounded Evolutionary Learning.

A Single-Task and Multi-Decision Evolutionary Game Model Based on Multi-Agent Reinforcement Learning

Evolutionary models in economics: a survey of methods and building blocks

Learning from Zero: How to Make Consumption-Saving Decisions in a Stochastic Environment with an AI Algorithm

Integrating Evolutionary Game-Theoretical Methods and Deep Reinforcement Learning for Adaptive Strategy Optimization in User-Side Electricity Markets: A Comprehensive Review

Oscillatory evolution of collective behavior in evolutionary games played with reinforcement learning

Evolutionary Reinforcement Learning: A Systematic Review and Future Directions

Evolutionary Algorithms for Reinforcement Learning

Auto-configuring Exploration-Exploitation Tradeoff in Evolutionary Computation via Deep Reinforcement Learning

Machine learning-based evolution model and the simulation of a profit model of agricultural products logistics financing

Evolutionary Reinforcement Learning: A Survey

Mathematics of multi-agent learning systems at the interface of game theory and artificial intelligence

Reinforcement Learning-assisted Evolutionary Algorithm: A Survey and Research Opportunities

Evolutionary Game Theory Squared: Evolving Agents in Endogenously Evolving Zero-Sum Games

Evolution with Opponent-Learning Awareness

Learning from Learning Machines: Optimisation, Rules, and Social Norms

Evolutionary Multi-Armed Bandits with Genetic Thompson Sampling

An electronic-game framework for evaluating coevolutionary algorithms