Abstract:In numerous artificial intelligence applications, the collaborative efforts of multiple intelligent agents are imperative for the successful attainment of target objectives. To enhance coordination among these agents, a distributed communication framework is often employed. However, indiscriminate information sharing among all agents can be resource-intensive, and the adoption of manually pre-defined communication architectures imposes constraints on inter-agent communication, thus limiting the potential for effective collaboration. Moreover, the communication framework often remains static during inference, which may result in sustained high resource consumption, as in most cases, only key decisions necessitate information sharing among agents. In this study, we introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph. We formulate this problem as the task of determining the communication graph while enabling the architecture parameters to update normally, thus necessitating a bi-level optimization process. Utilizing continuous relaxation of the graph representation and incorporating attention units, our proposed approach, CommFormer, efficiently optimizes the communication graph and concurrently refines architectural parameters through gradient descent in an end-to-end manner. Additionally, we introduce a temporal gating mechanism for each agent, enabling dynamic decisions on whether to receive shared information at a given time, based on current observations, thus improving decision-making efficiency. Extensive experiments on a variety of cooperative tasks substantiate the robustness of our model across diverse cooperative scenarios, where agents are able to develop more coordinated and sophisticated strategies regardless of changes in the number of agents.

Learning Cooperative Policies with Graph Networks in Distributed Swarm Systems.

Learning Intra-group Cooperation in Multi-agent Systems.

Learning Hierarchical Graph-Based Policy for Goal-Reaching in Unknown Environments

Cooperative Flocking And Learning In Multi-Robot Systems For Predator Avoidance

Learning of Coordination Policies for Robotic Swarms

Deep Hierarchical Communication Graph in Multi-Agent Reinforcement Learning.

Cooperative Policy Learning with Pre-trained Heterogeneous Observation Representations

Hierarchical RNNs with Graph Policy and Attention for Drone Swarm

Multi-Agent Actor-Critic with Hierarchical Graph Attention Network

Efficient Policy Generation in Multi-Agent Systems via Hypergraph Neural Network

Policy Consensus-Based Distributed Deterministic Multi-Agent Reinforcement Learning over Directed Graphs

Learning Multi-Agent Communication from Graph Modeling Perspective

Learning Decentralized Flocking Controllers with Spatio-Temporal Graph Neural Network

A Cooperation Graph Approach for Multiagent Sparse Reward Reinforcement Learning

Learning to Dynamically Coordinate Multi-Robot Teams in Graph Attention Networks

Multi-Agent Game Abstraction Via Graph Attention Neural Network.

Scalable and Transferable Reinforcement Learning for Multi-Agent Mixed Cooperative–Competitive Environments Based on Hierarchical Graph Attention

Communication Learning in Multi-Agent Systems from Graph Modeling Perspective

Hierarchical Reinforcement Learning with Opponent Modeling for Distributed Multi-agent Cooperation

Cooperative Planning of Multi-Uav Logistics Delivery by Multi-Graph Reinforcement Learning

Multi-Agent Reinforcement Learning for Distributed Cooperative Targets Search