Abstract:In numerous artificial intelligence applications, the collaborative efforts of multiple intelligent agents are imperative for the successful attainment of target objectives. To enhance coordination among these agents, a distributed communication framework is often employed. However, indiscriminate information sharing among all agents can be resource-intensive, and the adoption of manually pre-defined communication architectures imposes constraints on inter-agent communication, thus limiting the potential for effective collaboration. Moreover, the communication framework often remains static during inference, which may result in sustained high resource consumption, as in most cases, only key decisions necessitate information sharing among agents. In this study, we introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph. We formulate this problem as the task of determining the communication graph while enabling the architecture parameters to update normally, thus necessitating a bi-level optimization process. Utilizing continuous relaxation of the graph representation and incorporating attention units, our proposed approach, CommFormer, efficiently optimizes the communication graph and concurrently refines architectural parameters through gradient descent in an end-to-end manner. Additionally, we introduce a temporal gating mechanism for each agent, enabling dynamic decisions on whether to receive shared information at a given time, based on current observations, thus improving decision-making efficiency. Extensive experiments on a variety of cooperative tasks substantiate the robustness of our model across diverse cooperative scenarios, where agents are able to develop more coordinated and sophisticated strategies regardless of changes in the number of agents.

Multi-Agent Sequential Decision-Making via Communication

Multi-Agent Coordination via Multi-Level Communication

Communication Decision in Decentralized Control of Coordinated System

Research on Decentralized Communication Decision in the Multi-Agent Robotic System

Communication Learning in Multi-Agent Systems from Graph Modeling Perspective

Sequential Asynchronous Action Coordination in Multi-Agent Systems: A Stackelberg Decision Transformer Approach

Team-wise effective communication in multi-agent reinforcement learning

Cooperative Multi-agent Reinforcement Learning with Hierachical Communication Architecture

Multi-agent Coordination Under Temporal Logic Tasks and Team-Wise Intermittent Communication

Research on Multi-Agent Communication and Collaborative Decision-Making Based on Deep Reinforcement Learning

Learning Selective Communication for Multi-Agent Path Finding

RoundTable: Investigating Group Decision-Making Mechanism in Multi-Agent Collaboration

Autonomous Communication Decision Making Based on Graph Convolution Neural Network.

Learning Multi-Agent Communication from Graph Modeling Perspective

Modeling Communication of Collaborative Multi-Agent System under Epistemic Planning

Learning Multi-Agent Communication with Contrastive Learning

A survey of multi-agent deep reinforcement learning with communication

Optimistic sequential multi-agent reinforcement learning with motivational communication

Multi-Agent Curricula and Emergent Implicit Signaling

Improving Coordination in Small-Scale Multi-Agent Deep Reinforcement Learning through Memory-driven Communication

A Decentralized Communication Framework based on Dual-Level Recurrence for Multi-Agent Reinforcement Learning