Abstract:Multi-Agent Path Finding (MAPF) is a critical component of logistics and warehouse management, which focuses on planning collision-free paths for a team of robots in a known environment. Recent work introduced a novel MAPF approach, LNS2, which proposed to repair a quickly-obtainable set of infeasible paths via iterative re-planning, by relying on a fast, yet lower-quality, priority-based planner. At the same time, there has been a recent push for Multi-Agent Reinforcement Learning (MARL) based MAPF algorithms, which let agents learn decentralized policies that exhibit improved cooperation over such priority planning, although inevitably remaining slower. In this paper, we introduce a new MAPF algorithm, LNS2+RL, which combines the distinct yet complementary characteristics of LNS2 and MARL to effectively balance their individual limitations and get the best from both worlds. During early iterations, LNS2+RL relies on MARL for low-level re-planning, which we show eliminates collisions much more than a priority-based planner. There, our MARL-based planner allows agents to reason about past and future/predicted information to gradually learn cooperative decision-making through a finely designed curriculum learning. At later stages of planning, LNS2+RL adaptively switches to priority-based planning to quickly resolve the remaining collisions, naturally trading-off solution quality and computational efficiency. Our comprehensive experiments on challenging tasks across various team sizes, world sizes, and map structures consistently demonstrate the superior performance of LNS2+RL compared to many MAPF algorithms, including LNS2, LaCAM, and EECBS, where LNS2+RL shows significantly better performance in complex scenarios. We finally experimentally validate our algorithm in a hybrid simulation of a warehouse mockup involving a team of 100 (real-world and simulated) robots.

HELSA: Hierarchical Reinforcement Learning with Spatiotemporal Abstraction for Large-Scale Multi-Agent Path Finding

Learning to Cooperate: Application of Deep Reinforcement Learning for Online AGV Path Finding.

Moving Forward in Formation: A Decentralized Hierarchical Learning Approach to Multi-Agent Moving Together

Multi-Agent Path Finding via Reinforcement Learning with Hybrid Reward

HiMAP: Learning Heuristics-Informed Policies for Large-Scale Multi-Agent Pathfinding

Multi-Agent Path Finding Method Based on Evolutionary Reinforcement Learning

Curriculum Learning Based Multi-Agent Path Finding for Complex Environments.

Hierarchical Coordination Multi-Agent Reinforcement Learning with Spatio-Temporal Abstraction

Hierarchical Deep Multiagent Reinforcement Learning with Temporal Abstraction

Cooperative Reward Shaping for Multi-Agent Pathfinding

High-Sample-Efficient Multiagent Reinforcement Learning for Navigation and Collision Avoidance of UAV Swarms in Multitask Environments

SCRIMP: Scalable Communication for Reinforcement- and Imitation-Learning-Based Multi-Agent Pathfinding

Ensembling Prioritized Hybrid Policies for Multi-agent Pathfinding

Multi-Robot Path Planning Combining Heuristics and Multi-Agent Reinforcement Learning

Hierarchical reinforcement learning for handling sparse rewards in multi-goal navigation

RDE: A Hybrid Policy Framework for Multi-Agent Path Finding Problem

Deep Multi-Agent Reinforcement Learning with Discrete-Continuous Hybrid Action Spaces

Learning team-based navigation: a review of deep reinforcement learning techniques for multi-agent pathfinding

Multi-agent Pathfinding with Local and Global Guidance

Hierarchical Task Network Planning for Facilitating Cooperative Multi-Agent Reinforcement Learning

LNS2+RL: Combining Multi-agent Reinforcement Learning with Large Neighborhood Search in Multi-agent Path Finding