Abstract:With the breakthrough of AlphaGo, deep reinforcement learning has become a recognized technique for solving sequential decision-making problems. Despite its reputation, data inefficiency caused by its trial and error learning mechanism makes deep reinforcement learning difficult to apply in a wide range of areas. Many methods have been developed for sample efficient deep reinforcement learning, such as environment modelling, experience transfer, and distributed modifications, among which distributed deep reinforcement learning has shown its potential in various applications, such as human-computer gaming and intelligent transportation. In this paper, we conclude the state of this exciting field, by comparing the classical distributed deep reinforcement learning methods and studying important components to achieve efficient distributed learning, covering single player single agent distributed deep reinforcement learning to the most complex multiple players multiple agents distributed deep reinforcement learning. Furthermore, we review recently released toolboxes that help to realize distributed deep reinforcement learning without many modifications of their non-distributed versions. By analysing their strengths and weaknesses, a multi-player multi-agent distributed deep reinforcement learning toolbox is developed and released, which is further validated on Wargame, a complex environment, showing the usability of the proposed toolbox for multiple players and multiple agents distributed deep reinforcement learning under complex games. Finally, we try to point out challenges and future trends, hoping that this brief review can provide a guide or a spark for researchers who are interested in distributed deep reinforcement learning.

Distributed Methods for Reinforcement Learning Survey

Observer-Based Multiagent Deep Reinforcement Learning: A Fully Distributed Training Scheme

Robot Simulation and Reinforcement Learning Training Platform Based on Distributed Architecture.

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

Distributed Deep Reinforcement Learning: A Survey and a Multi-player Multi-agent Learning Toolbox

Target-Value-Competition-Based Multi-Agent Deep Reinforcement Learning Algorithm for Distributed Nonconvex Economic Dispatch

Multiagent Reinforcement Learning for Strictly Constrained Tasks Based on Reward Recorder

Acceleration for Deep Reinforcement Learning using Parallel and Distributed Computing: A Survey

Fully Decentralized Cooperative Multi-Agent Reinforcement Learning: A Survey

Distributed Multi-Agent Reinforcement Learning Based on Graph-Induced Local Value Functions

Multi-Agent Deep Reinforcement Learning for Multi-Robot Applications: A Survey

Distributed Reinforcement Learning for Robot Teams: a Review

Communication-Efficient Policy Gradient Methods for Distributed Reinforcement Learning

Distributed Optimization Methods for Multi-Robot Systems: Part II -- A Survey

Distributed Optimization Methods for Multi-robot Systems: Part 2—A Survey

A Survey From Distributed Machine Learning to Distributed Deep Learning

A survey on multi-agent deep reinforcement learning: from the perspective of challenges and applications

Efficient Distributed Framework for Collaborative Multi-Agent Reinforcement Learning

Toward Multi-Agent Reinforcement Learning for Distributed Event-Triggered Control

A Survey on Population-Based Deep Reinforcement Learning

Scalable and Sample Efficient Distributed Policy Gradient Algorithms in Multi-Agent Networked Systems