Abstract:In numerous artificial intelligence applications, the collaborative efforts of multiple intelligent agents are imperative for the successful attainment of target objectives. To enhance coordination among these agents, a distributed communication framework is often employed. However, indiscriminate information sharing among all agents can be resource-intensive, and the adoption of manually pre-defined communication architectures imposes constraints on inter-agent communication, thus limiting the potential for effective collaboration. Moreover, the communication framework often remains static during inference, which may result in sustained high resource consumption, as in most cases, only key decisions necessitate information sharing among agents. In this study, we introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph. We formulate this problem as the task of determining the communication graph while enabling the architecture parameters to update normally, thus necessitating a bi-level optimization process. Utilizing continuous relaxation of the graph representation and incorporating attention units, our proposed approach, CommFormer, efficiently optimizes the communication graph and concurrently refines architectural parameters through gradient descent in an end-to-end manner. Additionally, we introduce a temporal gating mechanism for each agent, enabling dynamic decisions on whether to receive shared information at a given time, based on current observations, thus improving decision-making efficiency. Extensive experiments on a variety of cooperative tasks substantiate the robustness of our model across diverse cooperative scenarios, where agents are able to develop more coordinated and sophisticated strategies regardless of changes in the number of agents.

Graphcomm: a graph neural network based method for multi-agent reinforcement learning

Graph Neural Network Meets Multi-Agent Reinforcement Learning: Fundamentals, Applications, and Future Directions

HyperComm: Hypergraph-based communication in multi-agent reinforcement learning

GraphComm: Efficient Graph Convolutional Communication for Multiagent Cooperation

Multiagent Reinforcement Learning With Graphical Mutual Information Maximization

Robust Multi-Agent Communication With Graph Information Bottleneck Optimization

Group-Aware Coordination Graph for Multi-Agent Reinforcement Learning

Learning Multi-Agent Communication from Graph Modeling Perspective

A Cooperation Graph Approach for Multiagent Sparse Reward Reinforcement Learning

Scaling Up Multiagent Reinforcement Learning for Robotic Systems: Learn an Adaptive Sparse Communication Graph

Multiagent Reinforcement Learning With Heterogeneous Graph Attention Network

Learning Efficient and Robust Multi-Agent Communication Via Graph Information Bottleneck

Graph Neural Networks with Model-based Reinforcement Learning for Multi-agent Systems

Graph Neural Network-based Multi-agent Reinforcement Learning for Resilient Distributed Coordination of Multi-Robot Systems

Communication Learning in Multi-Agent Systems from Graph Modeling Perspective

Multi-Agent Game Abstraction Via Graph Attention Neural Network.

Heterogeneous Multi-Agent Communication Learning Via Graph Information Maximization.

Graph MADDPG with RNN for multiagent cooperative environment

Learning Attentional Communication with a Common Network for Multiagent Reinforcement Learning.

Enhancing Collaboration in Heterogeneous Multiagent Systems Through Communication Complementary Graph

FCMNet: Full Communication Memory Net for Team-Level Cooperation in Multi-Agent Systems