Abstract:With the breakthrough of AlphaGo, deep reinforcement learning has become a recognized technique for solving sequential decision-making problems. Despite its reputation, data inefficiency caused by its trial and error learning mechanism makes deep reinforcement learning difficult to apply in a wide range of areas. Many methods have been developed for sample efficient deep reinforcement learning, such as environment modelling, experience transfer, and distributed modifications, among which distributed deep reinforcement learning has shown its potential in various applications, such as human-computer gaming and intelligent transportation. In this paper, we conclude the state of this exciting field, by comparing the classical distributed deep reinforcement learning methods and studying important components to achieve efficient distributed learning, covering single player single agent distributed deep reinforcement learning to the most complex multiple players multiple agents distributed deep reinforcement learning. Furthermore, we review recently released toolboxes that help to realize distributed deep reinforcement learning without many modifications of their non-distributed versions. By analysing their strengths and weaknesses, a multi-player multi-agent distributed deep reinforcement learning toolbox is developed and released, which is further validated on Wargame, a complex environment, showing the usability of the proposed toolbox for multiple players and multiple agents distributed deep reinforcement learning under complex games. Finally, we try to point out challenges and future trends, hoping that this brief review can provide a guide or a spark for researchers who are interested in distributed deep reinforcement learning.

DeepMind Lab2D

DeepMind Lab

Psychlab: A Psychology Laboratory for Deep Reinforcement Learning Agents

D2RL: Deep Dense Architectures in Reinforcement Learning

DIAMBRA Arena: a New Reinforcement Learning Platform for Research and Experimentation

Toybox: A Suite of Environments for Experimental Evaluation of Deep Reinforcement Learning

Exploring Exploration: Comparing Children with RL Agents in Unified Environments

DeepSim: A Reinforcement Learning Environment Build Toolkit for ROS and Gazebo

The Animal-AI Environment: A Virtual Laboratory For Comparative Cognition and Artificial Intelligence Research

Deep Reinforcement Learning Boosted by External Knowledge

Google Research Football: A Novel Reinforcement Learning Environment

Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAX

Deep Surrogate Assisted Generation of Environments

Scaling Instructable Agents Across Many Simulated Worlds

Deep Apprenticeship Learning for Playing Games

Deep Reinforcement Learning-based Large-scale Robot Exploration

Multi-Agent Deep Reinforcement Learning with Human Strategies

Distributed Deep Reinforcement Learning: A Survey and a Multi-player Multi-agent Learning Toolbox

Novel Reinforcement Learning Research Platform for Role-Playing Games

Development of a Simulation Environment for Robot Soccer Game with Deep Reinforcement Learning and Role Assignment

Megaverse: Simulating Embodied Agents at One Million Experiences per Second