Abstract:The mainstream Multi-Agent Reinforcement Learning (MARL) methods introduce the teammate modeling or the communication mechanism into Centralized Training Decentralized Execution (CTDE) paradigm, which can improve coordination performance. However, the existing teammate modeling methods predict either actions or local observations, limiting their applicability. In addition, the traditional communication mechanism only considers the quantity of the communication links while ignoring the quality of retained communication links, leading to inefficient and redundant communication. To solve the above problems, this paper proposes a novel Multi-Agent Cooperative Strategy with Explicit Teammate Modeling and Targeted Informative Communication (MACS), which can generate and send the more informative message with the higher communication efficiency, further improving the coordination performance. Specifically, the Variational Auto-Encoder (VAE) is leveraged to allow each agent to simultaneously predict the observations and actions of teammates, thus generating more comprehensive communication message. Then, we propose a new Mutual Information (MI) between the communication message and teammate Q-value, which can obtain the informative message, ensuring the exploration and stability of the method. In addition, a targeted dynamic informative communication graph is established by the Graph Neural Network (GNN) which can reduce the redundant communication link through hypothetical analysis, further improving the overall communication efficiency. Eventually, we conduct experiments in StarCraft II, Collaborative Navigation, and Multi-Target Multi-Sensor Coverage environments. Experimental results show that the proposed approach is superior to the state-of-the-art in terms of coordination performance and communication efficiency.

Adaptive Reward Method for End-to-End Cooperation Based on Multi-agent Reinforcement Learning

Learning Intra-group Cooperation in Multi-agent Systems.

A Graph-Based Soft Actor Critic Approach in Multi-Agent Reinforcement Learning

A Cooperation Graph Approach for Multiagent Sparse Reward Reinforcement Learning

Credit assignment in heterogeneous multi-agent reinforcement learning for fully cooperative tasks

Multi-Agent Feature Learning and Integration for Mixed Cooperative and Competitive Environment.

Multi-Level Credit Assignment for Cooperative Multi-Agent Reinforcement Learning

Situation-Dependent Causal Influence-Based Cooperative Multi-agent Reinforcement Learning

A Cooperative Multi-Agent Reinforcement Learning Method Based on Coordination Degree

Optimistic sequential multi-agent reinforcement learning with motivational communication

Priority over Quantity: A Self-Incentive Credit Assignment Scheme for Cooperative Multiagent Reinforcement Learning

Inducing Cooperation via Team Regret Minimization based Multi-Agent Deep Reinforcement Learning

Promoting Cooperation in Multi-Agent Reinforcement Learning via Mutual Help

Credible Negotiation for Multi-agent Reinforcement Learning in Long-term Coordination

Multi-agent Cooperative Strategy with Explicit Teammate Modeling and Targeted Informative Communication

Multiagent Reinforcement Learning With Heterogeneous Graph Attention Network

Cooperative Policy Learning with Pre-trained Heterogeneous Observation Representations

Enhancing Heterogeneous Multi-Agent Cooperation in Decentralized MARL via GNN-driven Intrinsic Rewards

Closely Cooperative Multi-Agent Reinforcement Learning Based on Intention Sharing and Credit Assignment

Hierarchical Consensus-Based Multi-Agent Reinforcement Learning for Multi-Robot Cooperation Tasks

Peer Incentive Reinforcement Learning for Cooperative Multiagent Games