Abstract:Deep reinforcement learning has made significant progress in multi-agent tasks in recent years. However, most previous studies focus on solving fully cooperative tasks, which do not perform well in mixed tasks. In mixed tasks, the agent needs to comprehensively consider the information provided by its friends and enemies to learn its strategy, and its strategy is sensitive to the received information. Additionally, the input space of the critic network increases rapidly with the number of agents in the actor-critic framework. It’s of great necessity to efficiently learn information representation to obtain important features. To this end, we present an approach that conducts information representation with attention mechanism. Our approach adopts the framework of centralized training and decentralized execution. We apply the multi-head hierarchical attention mechanism to centrally computed critics, so critics can process the received information more accurately and assist actors in choosing better actions. The hierarchical attention critic adopts a bi-level attention structure which is composed of the agent-level and the group-level. They are designed to assign different weights to friends’ and enemies’ information and then summarize them at each timestep. It achieves high efficiency and scalability in mixed tasks. Furthermore, we use the feature extraction based on the recurrent neural network to encode the state-action sequence information of each agent. Experimental results show that our approach is not only applicable to cooperative environments but also better in mixed environments, especially in the predator-prey task, the reward obtained by our method is twice that of the baselines.

AHAC: Actor Hierarchical Attention Critic for Multi-Agent Reinforcement Learning.

Multi actor hierarchical attention critic with RNN-based feature extraction

Multi-Agent Actor-Critic with Hierarchical Graph Attention Network

Scalable and Transferable Reinforcement Learning for Multi-Agent Mixed Cooperative–Competitive Environments Based on Hierarchical Graph Attention

Learning Multi-Agent Communication with Double Attentional Deep Reinforcement Learning

Learning Attentional Communication with a Common Network for Multiagent Reinforcement Learning.

Complementary Attention for Multi-Agent Reinforcement Learning.

Large-scale UAV swarm confrontation based on hierarchical attention actor-critic algorithm

Meta Actor-Critic Framework for Multi-Agent Reinforcement Learning

Cooperative multi-agent game based on reinforcement learning

Subgoal-based Hierarchical Reinforcement Learning for Multi-Agent Collaboration

Multi-agent Reinforcement Learning with Multi-head Attention

Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments

Cascaded Attention: Adaptive and Gated Graph Attention Network for Multiagent Reinforcement Learning

Inner Attention Supported Adaptive Cooperation for Heterogeneous Multi Robots Teaming based on Multi-agent Reinforcement Learning

Collaborative Decision-Making Method for Multi-UAV Based on Multiagent Reinforcement Learning

An Actor-Critic-Attention Mechanism for Deep Reinforcement Learning in Multi-view Environments

Hierarchical Reinforcement Learning with Opponent Modeling for Distributed Multi-agent Cooperation

Attention Enhanced Reinforcement Learning for Multi agent Cooperation

Multiagent Reinforcement Learning With Heterogeneous Graph Attention Network

Deep Hierarchical Communication Graph in Multi-Agent Reinforcement Learning.