Abstract:Deep reinforcement learning has made significant progress in multi-agent tasks in recent years. However, most previous studies focus on solving fully cooperative tasks, which do not perform well in mixed tasks. In mixed tasks, the agent needs to comprehensively consider the information provided by its friends and enemies to learn its strategy, and its strategy is sensitive to the received information. Additionally, the input space of the critic network increases rapidly with the number of agents in the actor-critic framework. It’s of great necessity to efficiently learn information representation to obtain important features. To this end, we present an approach that conducts information representation with attention mechanism. Our approach adopts the framework of centralized training and decentralized execution. We apply the multi-head hierarchical attention mechanism to centrally computed critics, so critics can process the received information more accurately and assist actors in choosing better actions. The hierarchical attention critic adopts a bi-level attention structure which is composed of the agent-level and the group-level. They are designed to assign different weights to friends’ and enemies’ information and then summarize them at each timestep. It achieves high efficiency and scalability in mixed tasks. Furthermore, we use the feature extraction based on the recurrent neural network to encode the state-action sequence information of each agent. Experimental results show that our approach is not only applicable to cooperative environments but also better in mixed environments, especially in the predator-prey task, the reward obtained by our method is twice that of the baselines.

Meta Attention for Off-Policy Actor-Critic.

Online Meta-Critic Learning for Off-Policy Actor-Critic Methods

Meta Actor-Critic Framework for Multi-Agent Reinforcement Learning

MAPPO method based on attention behavior network

Multi actor hierarchical attention critic with RNN-based feature extraction

An Actor-Critic-Attention Mechanism for Deep Reinforcement Learning in Multi-view Environments

A Task-Aware Attention-Based Method for Improved Meta-Learning.

Inverse Attention Agent for Multi-Agent System

A priority experience replay actor-critic algorithm using self-attention mechanism for strategy optimization of discrete problems

Complementary Attention for Multi-Agent Reinforcement Learning.

Multi-agent Reinforcement Learning with Multi-head Attention

Self-attention-based multi-agent continuous control method in cooperative environments

Multi-Agent Actor-Critic with Hierarchical Graph Attention Network

Better Deep Visual Attention with Reinforcement Learning in Action Recognition.

Learning to Learn: Meta-Critic Networks for Sample Efficient Learning.

Inner Attention Supported Adaptive Cooperation for Heterogeneous Multi Robots Teaming based on Multi-agent Reinforcement Learning

A Multi-Agent Off-Policy Actor-Critic Algorithm for Distributed Reinforcement Learning

Attention or memory? Neurointerpretable agents in space and time

Attention-Privileged Reinforcement Learning

Prioritized Experience Replay in Multi-Actor-Attention-Critic for Reinforcement Learning

Joint Attention for Multi-Agent Coordination and Social Learning