Abstract:In numerous artificial intelligence applications, the collaborative efforts of multiple intelligent agents are imperative for the successful attainment of target objectives. To enhance coordination among these agents, a distributed communication framework is often employed. However, indiscriminate information sharing among all agents can be resource-intensive, and the adoption of manually pre-defined communication architectures imposes constraints on inter-agent communication, thus limiting the potential for effective collaboration. Moreover, the communication framework often remains static during inference, which may result in sustained high resource consumption, as in most cases, only key decisions necessitate information sharing among agents. In this study, we introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph. We formulate this problem as the task of determining the communication graph while enabling the architecture parameters to update normally, thus necessitating a bi-level optimization process. Utilizing continuous relaxation of the graph representation and incorporating attention units, our proposed approach, CommFormer, efficiently optimizes the communication graph and concurrently refines architectural parameters through gradient descent in an end-to-end manner. Additionally, we introduce a temporal gating mechanism for each agent, enabling dynamic decisions on whether to receive shared information at a given time, based on current observations, thus improving decision-making efficiency. Extensive experiments on a variety of cooperative tasks substantiate the robustness of our model across diverse cooperative scenarios, where agents are able to develop more coordinated and sophisticated strategies regardless of changes in the number of agents.

Cooperative Multi-agent Reinforcement Learning with Hierachical Communication Architecture

Learning Intra-group Cooperation in Multi-agent Systems.

Moving Forward in Formation: A Decentralized Hierarchical Learning Approach to Multi-Agent Moving Together

Hierarchical Reinforcement Learning with Opponent Modeling for Distributed Multi-agent Cooperation

A Hierarchical Framework for Cooperative Tasks in Multi-agent Systems

Deep Hierarchical Communication Graph in Multi-Agent Reinforcement Learning.

Effective Master-Slave Communication On A Multi-Agent Deep Reinforcement Learning System

Learning Structured Communication for Multi-agent Reinforcement Learning

AC2C: Adaptively Controlled Two-Hop Communication for Multi-Agent Reinforcement Learning

Hierarchical Multi-Agent Reinforcement Learning for Cooperative Tasks with Sparse Rewards in Continuous Domain

Verco: Learning Coordinated Verbal Communication for Multi-agent Reinforcement Learning

Subgoal-based Hierarchical Reinforcement Learning for Multi-Agent Collaboration

Team-wise effective communication in multi-agent reinforcement learning

Multi-agent deep reinforcement learning with type-based hierarchical group communication

Communication Learning in Multi-Agent Systems from Graph Modeling Perspective

Hierarchical Reinforcement Learning Based Multi-Agent Collaborative Control Approach

Scalable and Transferable Reinforcement Learning for Multi-Agent Mixed Cooperative–Competitive Environments Based on Hierarchical Graph Attention

Multi-Agent Coordination via Multi-Level Communication

GHGC: Goal-based Hierarchical Group Communication in Multi-Agent Reinforcement Learning.

Learning Practical Communication Strategies in Cooperative Multi-Agent Reinforcement Learning

Modeling Sensorimotor Coordination as Multi-Agent Reinforcement Learning with Differentiable Communication