Abstract:Heterogeneous multiagent systems are characterized by diverse task distributions, which are prevalent in practical scenarios, such as distributed decision making and robotic collaboration. A significant challenge in these systems is the constraint of limited observations, where each agent has access only to partial information. Many studies facilitate information exchange by employing shared parameters among agents. However, this approach is generally more effective for homogeneous systems where agents have similar observation or action spaces. In heterogeneous systems, indiscriminate parameter sharing can significantly increase the exploration cost required for effective adaptation. To address this challenge, we propose a novel communication complementary graph model (CCGM) for enhancing collaboration in heterogeneous multiagent systems. Our approach builds upon the training framework of heterogeneous agent reinforcement learning (HARL) with trust region learning and nonparameter sharing. This model utilizes advantage function decomposition and sequential updates to promote policy convergence. Within this framework, we introduce a novel communication method inspired by signaling games, where agents acting as receivers, process messages from other agents alongside their own observations. CCGM aligns the messages with observations in a graph-based communication module, which establishes communication relationships and supplements observational information. Subsequently, agents generate self-interested information, which they then share with others as senders. We evaluate our algorithm across various environments, including multiagent particle environments (MPE) and multiagent MuJoCo (MAMuJoCo) robot experiments. The results demonstrate the effectiveness of CCGM in enhancing HARL-based algorithms.

HiSA: Facilitating Efficient Multi-Agent Coordination and Cooperation by Hierarchical Policy with Shared Attention

Learning Intra-group Cooperation in Multi-agent Systems.

S2rl

Learning Attentional Communication for Multi-Agent Cooperation

HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanism

Enhancing Multi-Agent Coordination through Common Operating Picture Integration

Hierarchical Consensus-Based Multi-Agent Reinforcement Learning for Multi-Robot Cooperation Tasks

Enhancing Collaboration in Heterogeneous Multiagent Systems Through Communication Complementary Graph

Attention Enhanced Reinforcement Learning for Multi agent Cooperation

Optimistic sequential multi-agent reinforcement learning with motivational communication

Subgoal-based Hierarchical Reinforcement Learning for Multi-Agent Collaboration

Learning Hierarchical Teaching Policies for Cooperative Agents

Scalable and Transferable Reinforcement Learning for Multi-Agent Mixed Cooperative–Competitive Environments Based on Hierarchical Graph Attention

Multi-Agent Incentive Communication via Decentralized Teammate Modeling

Multi-Agent Coordination via Multi-Level Communication

Efficient Multi-agent Communication via Self-supervised Information Aggregation

Hierarchical Reinforcement Learning with Opponent Modeling for Distributed Multi-agent Cooperation

Self-Clustering Hierarchical Multi-Agent Reinforcement Learning with Extensible Cooperation Graph

Leveraging Organizational Hierarchy to Simplify Reward Design in Cooperative Multi-agent Reinforcement Learning

Learning Multi-Agent Communication with Double Attentional Deep Reinforcement Learning

Coordination as inference in multi-agent reinforcement learning