Abstract:Heterogeneous multiagent systems are characterized by diverse task distributions, which are prevalent in practical scenarios, such as distributed decision making and robotic collaboration. A significant challenge in these systems is the constraint of limited observations, where each agent has access only to partial information. Many studies facilitate information exchange by employing shared parameters among agents. However, this approach is generally more effective for homogeneous systems where agents have similar observation or action spaces. In heterogeneous systems, indiscriminate parameter sharing can significantly increase the exploration cost required for effective adaptation. To address this challenge, we propose a novel communication complementary graph model (CCGM) for enhancing collaboration in heterogeneous multiagent systems. Our approach builds upon the training framework of heterogeneous agent reinforcement learning (HARL) with trust region learning and nonparameter sharing. This model utilizes advantage function decomposition and sequential updates to promote policy convergence. Within this framework, we introduce a novel communication method inspired by signaling games, where agents acting as receivers, process messages from other agents alongside their own observations. CCGM aligns the messages with observations in a graph-based communication module, which establishes communication relationships and supplements observational information. Subsequently, agents generate self-interested information, which they then share with others as senders. We evaluate our algorithm across various environments, including multiagent particle environments (MPE) and multiagent MuJoCo (MAMuJoCo) robot experiments. The results demonstrate the effectiveness of CCGM in enhancing HARL-based algorithms.

Partial Communication Model Based on the Gain of Q-value in Multi-agent Reinforcement Learning

Shapley Q-Value: A Local Reward Approach to Solve Global Reward Games

Learning Effective Communication for Cooperative Pursuit with Multi-Agent Reinforcement Learning

Enhancing Collaboration in Heterogeneous Multiagent Systems Through Communication Complementary Graph

Fully Independent Communication in Multi-Agent Reinforcement Learning

Q-value Path Decomposition for Deep Multiagent Reinforcement Learning

QPLEX: Duplex Dueling Multi-Agent Q-Learning.

Multi-Agent Incentive Communication via Decentralized Teammate Modeling

Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL?

Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learning

PMAC: Personalized Multi-Agent Communication

Reinforcement learning for encouraging cooperation in a multiagent system

Learning Nearly Decomposable Value Functions Via Communication Minimization

Efficient Communications in Multi-Agent Reinforcement Learning for Mobile Applications

Learning Multi-Agent Cooperation via Considering Actions of Teammates

Verco: Learning Coordinated Verbal Communication for Multi-agent Reinforcement Learning

Team-wise effective communication in multi-agent reinforcement learning

Effective Communications: A Joint Learning and Communication Framework for Multi-Agent Reinforcement Learning over Noisy Channels

RGMComm: Return Gap Minimization via Discrete Communications in Multi-Agent Reinforcement Learning

Optimistic sequential multi-agent reinforcement learning with motivational communication

Multiagent Q-learning with Sub-Team Coordination.