Abstract:Multi-agent deep reinforcement learning（MADRL） applies the ideas and algorithms of deep reinforcement learning to the learning and control of multi-agent systems, which is an important method to develop multi-agent systems with swarm agents. Existing MADRL studies mainly design algorithms based on the assumption that the environment is completely observable or communication resources are not limited. However, partial observability is an objective problem in the practical application of multi-agent systems. For example, the observation range of agentsis is usually limited, and the complete environmental information is not included outside the observable range, which makes it difficult for multiagent collaboration. Aiming at the problem of partial observability in real scenes, based on the paradigm of centralized training and distributed execution, this paper extends the deep reinforcement learning algorithm Actor-Critic to multiagent systems and adds communication channels and gating mechanisms between agents, finally proposes a recurrent gated multi-agent Actor-Critic（RGMAAC） algorithm. Agents can communicate efficiently based on the historical action observation sequence, and finally use the local observation, the historical observation sequence and observations shared by other agents through communication channels to make behavior decisions. Meanwhile, based on the multi-agent particle environment, the multi-agent task of synchronous and fast arrival is designed, and two reward value functions and task scenarios are designed respectively. The experimental results show that the trained agent with the RGMAAC algorithm performs well and is superior to the baseline algorithm in terms of stability when some observable problems clearly appear in the task scenario.

Macro-Action-Based Deep Multi-Agent Reinforcement Learning

Macro-Action-Based Multi-Agent/Robot Deep Reinforcement Learning under Partial Observability

Multi-Agent/Robot Deep Reinforcement Learning with Macro-Actions (Student Abstract)

Observer-Based Multiagent Deep Reinforcement Learning: A Fully Distributed Training Scheme

Learning for Decentralized Control of Multiagent Systems in Large, Partially-Observable Stochastic Environments

HiMacMic: Hierarchical Multi-Agent Deep Reinforcement Learning with Dynamic Asynchronous Macro Strategy

Deep Reinforcement Learning With Macro-Actions

Dueling Network Architecture for Multi-Agent Deep Deterministic Policy Gradient

Decentralized control of multi-robot partially observable Markov decision processes using belief space macro-actions

Deep Multi-Agent Reinforcement Learning with Discrete-Continuous Hybrid Action Spaces

Decentralized Control of Partially Observable Markov Decision Processes using Belief Space Macro-actions

A further exploration of deep Multi-Agent Reinforcement Learning with Hybrid Action Space

Multi-agent Deep Reinforcement Learning Based on Maximum Entropy

Decentralized Multi-Agent Reinforcement Learning: An Off-Policy Method

Macro Action Reinforcement Learning with Sequence Disentanglement using Variational Autoencoder

An Alternative Curriculum Learning Approach with Macro Actions for Deep Reinforcement Learning

Multi-agent Reinforcement Learning with Deep Networks for Diverse Q-Vectors

Effective Multi-Agent Deep Reinforcement Learning Control with Relative Entropy Regularization

Multi-agent collaboration based on RGMAAC algorithm under partial observability

On Improving Deep Reinforcement Learning for POMDPs

R-MADDPG for Partially Observable Environments and Limited Communication