Abstract:We investigate the problem of decentralized multi-agent navigation tasks, where multiple agents need to reach initially unassigned targets in a limited time. Classical planning-based methods suffer from expensive computation overhead at each step and offer limited expressiveness for complex cooperation strategies. In contrast, reinforcement learning (RL) has recently become a popular paradigm for addressing this issue. However, RL struggles with low data efficiency and cooperation when directly exploring (nearly) optimal policies in the large search space, especially with an increased agent number (e.g., 10+ agents) or in complex environments (e.g., 3D simulators). In this paper, we propose Multi-Agent Scalable GNN-based P lanner (MASP), a goal-conditioned hierarchical planner for navigation tasks with a substantial number of agents. MASP adopts a hierarchical framework to divide a large search space into multiple smaller spaces, thereby reducing the space complexity and accelerating training convergence. We also leverage graph neural networks (GNN) to model the interaction between agents and goals, improving goal achievement. Besides, to enhance generalization capabilities in scenarios with unseen team sizes, we divide agents into multiple groups, each with a previously trained number of agents. The results demonstrate that MASP outperforms classical planning-based competitors and RL baselines, achieving a nearly 100% success rate with minimal training data in both multi-agent particle environments (MPE) with 50 agents and a quadrotor 3-dimensional environment (OmniDrones) with 20 agents. Furthermore, the learned policy showcases zero-shot generalization across unseen team sizes.

Graph Neural Networks with Model-based Reinforcement Learning for Multi-agent Systems

Multi-Agent Trajectory Prediction Based on Graph Neural Network

A Graph-Based Soft Actor Critic Approach in Multi-Agent Reinforcement Learning

Graph Neural Network Meets Multi-Agent Reinforcement Learning: Fundamentals, Applications, and Future Directions

MASP: Scalable GNN-based Planning for Multi-Agent Navigation

Graphcomm: a graph neural network based method for multi-agent reinforcement learning

Multi-Agent Game Abstraction Via Graph Attention Neural Network.

MACNS: A Generic Graph Neural Network Integrated Deep Reinforcement Learning Based Multi-Agent Collaborative Navigation System for Dynamic Trajectory Planning

Graph Neural Network-based Multi-agent Reinforcement Learning for Resilient Distributed Coordination of Multi-Robot Systems

Graph Convolution-Based Deep Reinforcement Learning for Multi-Agent Decision-Making in Interactive Traffic Scenarios.

Graph MADDPG with RNN for multiagent cooperative environment

Deep Multi-Agent Reinforcement Learning with Relevance Graphs

Decentralized, Unlabeled Multi-Agent Navigation in Obstacle-Rich Environments Using Graph Neural Networks

Graph-based Multi-Agent Reinforcement Learning for Collaborative Search and Tracking of Multiple UAVs

Multiagent Reinforcement Learning With Heterogeneous Graph Attention Network

Multiagent Reinforcement Learning With Graphical Mutual Information Maximization

Multi-Agent Reinforcement Learning Algorithm Based On Neural Networks

MAGNet: Multi-agent Graph Network for Deep Multi-agent Reinforcement Learning

Attention Graph for Multi-Robot Social Navigation with Deep Reinforcement Learning

Soft-HGRNs: Soft Hierarchical Graph Recurrent Networks for Multi-Agent Partially Observable Environments

Analyzing Behavior and Intention in Multi-Agent Systems Using Graph Neural Networks