Abstract:The mainstream Multi-Agent Reinforcement Learning (MARL) methods introduce the teammate modeling or the communication mechanism into Centralized Training Decentralized Execution (CTDE) paradigm, which can improve coordination performance. However, the existing teammate modeling methods predict either actions or local observations, limiting their applicability. In addition, the traditional communication mechanism only considers the quantity of the communication links while ignoring the quality of retained communication links, leading to inefficient and redundant communication. To solve the above problems, this paper proposes a novel Multi-Agent Cooperative Strategy with Explicit Teammate Modeling and Targeted Informative Communication (MACS), which can generate and send the more informative message with the higher communication efficiency, further improving the coordination performance. Specifically, the Variational Auto-Encoder (VAE) is leveraged to allow each agent to simultaneously predict the observations and actions of teammates, thus generating more comprehensive communication message. Then, we propose a new Mutual Information (MI) between the communication message and teammate Q-value, which can obtain the informative message, ensuring the exploration and stability of the method. In addition, a targeted dynamic informative communication graph is established by the Graph Neural Network (GNN) which can reduce the redundant communication link through hypothetical analysis, further improving the overall communication efficiency. Eventually, we conduct experiments in StarCraft II, Collaborative Navigation, and Multi-Target Multi-Sensor Coverage environments. Experimental results show that the proposed approach is superior to the state-of-the-art in terms of coordination performance and communication efficiency.

Cooperative multi-agent game based on reinforcement learning

A Graph-Based Soft Actor Critic Approach in Multi-Agent Reinforcement Learning

Efficient Multi-Agent Exploration with Mutual-Guided Actor-Critic

Deep reinforcement learning algorithm based on multi-agent parallelism and its application in game environment

Decomposed Soft Actor-Critic Method for Cooperative Multi-Agent Reinforcement Learning

Cooperative Multi-Agent Reinforcement Learning with Hypergraph Convolution

Group-Aware Coordination Graph for Multi-Agent Reinforcement Learning

ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependency

APC: Predict Global Representation from Local Observation in Multi-Agent Reinforcement Learning

Multi-agent Cooperative Strategy with Explicit Teammate Modeling and Targeted Informative Communication

Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments

SC-MAIRL: Semi-Centralized Multi-Agent Imitation Reinforcement Learning

Learning Attentional Communication with a Common Network for Multiagent Reinforcement Learning.

HiMacMic: Hierarchical Multi-Agent Deep Reinforcement Learning with Dynamic Asynchronous Macro Strategy

Consciousness-Aware Multi-Agent Reinforcement Learning

Priority over Quantity: A Self-Incentive Credit Assignment Scheme for Cooperative Multiagent Reinforcement Learning

GAMA: Graph Attention Multi-agent reinforcement learning algorithm for cooperation

AHAC: Actor Hierarchical Attention Critic for Multi-Agent Reinforcement Learning.

F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learning

Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition

Attention-Guided Contrastive Role Representations for Multi-Agent Reinforcement Learning