AI Buzzwords Explained: Multi-Agent Path Finding (MAPF)

Hang Ma,Sven Koenig

DOI: https://doi.org/10.1145/3137574.3137579

2017-10-17

Abstract:Explanation of the hot topic "multi-agent path finding".

Artificial Intelligence,Multiagent Systems,Robotics

What problem does this paper attempt to address?

Overview: Generalizations of Multi-Agent Path Finding to Real-World Scenarios

Hang Ma,Sven Koenig,Nora Ayanian,Liron Cohen,Wolfgang Hoenig,T. K. Satish Kumar,Tansel Uras,Hong Xu,Craig Tovey,Guni Sharon

DOI: https://doi.org/10.48550/arXiv.1702.05515

2017-02-18

Abstract:Multi-agent path finding (MAPF) is well-studied in artificial intelligence, robotics, theoretical computer science and operations research. We discuss issues that arise when generalizing MAPF methods to real-world scenarios and four research directions that address them. We emphasize the importance of addressing these issues as opposed to developing faster methods for the standard formulation of the MAPF problem.

Artificial Intelligence,Multiagent Systems,Robotics
Multi-Agent Path Finding with heterogeneous edges and roundtrips

Bing Ai,Jiuchuan Jiang,Shoushui Yu,Yichuan Jiang

DOI: https://doi.org/10.1016/j.knosys.2021.107554

IF: 8.139

2021-01-01

Knowledge-Based Systems

Abstract:Multi-Agent Path Finding (MAPF) aims to find a set of conflict-free paths for multiple agents on a given graph from parking locations to specified goal locations while optimizing related costs. Currently, existing MAPF studies often consider the simplified problem setup where each agent does not return to its parking location after completing its task on the underlying graph with uniform edge costs. Nevertheless, within some real-word scenarios such as the Unmanned Aircraft System (UAS), agents are situated in the underlying graph with non-uniform edge costs. These agents are required to travel from their respective parking locations to complete tasks, and then return without conflicts. Therefore, this paper explores a new version of MAPF, formally called Multi-Agent Path Finding with Heterogeneous edges and Roundtrips (MAPF-HR). In this version, all agents are engaged in completing tasks by navigating their respective conflict-free paths with roundtrips on the graph with heterogeneous edges. This paper investigates a novel algorithm for this problem, called Improved Conflict-Based Search (CBS) with Helpful Bypass (ICBS-HB), which improves the CBS framework by utilizing the scheme of bypassing conflicts during path finding execution. With extensive experiments on MAPF benchmark maps, it shows that ICBS-HB outperforms the state-of-the-art algorithms for dealing with MAPF-HR.
Learning to Cooperate: Application of Deep Reinforcement Learning for Online AGV Path Finding.

Yi Zhang,Yu Qian,Yichen Yao,Haoyuan Hu,Yinghui Xu

2020-01-01

Abstract:Multi-agent path finding (MAPF), naturally exists in applications like picking-up and dropping-off parcels by automated guided vehicles (AGVs) in the warehouse. Existing algorithms, like conflict-based search (CBS), windowed hierarchical cooperative A* (WHCA), and other A* variants, are widely used to find the shortest paths in different manners. However, in real-world environments, MAPF cases are dynamically generated and need to be solved in real time. In this work, a decentralized multi-agent reinforcement learning (MARL) framework with multi-step ahead tree search (MATS) strategy is proposed to make efficient decisions. Through performing experiments on a 30*30 grid world and a real-world warehouse case, our proposed MARL policy is proved to be capable of: 1) scaling to a large number of agents in real-world environment with online response time within acceptable levels; 2) outperforming existing algorithms with shorter path length and solution time, as the number of agents increases.
Multi-Agent Path Finding with Heterogeneous Geometric and Kinematic Constraints in Continuous Space

Wenbo Lin,Wei Song,Qiuguo Zhu,Shiqiang Zhu

DOI: https://doi.org/10.1109/lra.2024.3511435

IF: 5.2

2024-01-01

IEEE Robotics and Automation Letters

Abstract:Multi-Agent Path Finding (MAPF) represents a pivotal area of research within multi-agent systems. Existing algorithms typically discretize the movement space of agents into grid or topology, neglecting agents' geometric characteristics and kinematic constraints. This limitation hampers their applicability and efficiency in practical industrial scenarios. In this paper, we propose a Priority-Based Search algorithm for heterogeneous mobile robots working in continuous space, addressing both geometric and kinematic constraints. This algorithm, named Continuous-space Heterogeneous Priority-Based Search (CHPBS), employs a two-level search structure and a priority tree for collision detection. To expedite single-agent path finding in continuous space, we introduce a Weighted Hybrid Safe Interval Path Planning algorithm (WHSIPP $_{d}$ ). Furthermore, we present three strategies to enhance our algorithm, collectively termed Enhanced-CHPBS (ECHPBS): Partial Expansion, Target Reasoning, and Adaptive Induced Priority. Comparative analysis against two baseline algorithms on a specialized benchmark demonstrates that ECHPBS achieves a success rate of 100% on a 100 m × 100 m map featuring 50 agents, with an average runtime of under 1 second, and maintains the same 100% success rate on a 300 m × 300 m map with 100 agents.
MAPF and MAPD: Recent Developments and Future Directions (short paper)

Benedetta Flammini,Davide Azzalini,Nicola Basilico,F. Amigoni

Abstract:Multi-Agent Path Finding (MAPF) is the problem of computing collision-free paths for a group of agents such that they can safely reach their target locations. In Multi-Agent Pickup and Delivery (MAPD), pickup and delivery locations are provided at runtime for an ideally ever-ending stream of tasks, making MAPD a combination between MAPF and online task assignment. Current models for MAPF and MAPD do not consider many of the practical issues encountered in real applications: in this paper, we present four new variants of the MAPF and MAPD problems we have recently devised which try to overcome different shortcomings of the original formulations. Promising directions of future work are also identified.

Computer Science
ALPHA: Attention-based Long-horizon Pathfinding in Highly-structured Areas

Chengyang He,Tianze Yang,Tanishq Duhan,Yutong Wang,Guillaume Sartoretti

DOI: https://doi.org/10.48550/arXiv.2310.08350

2023-10-12

Abstract:The multi-agent pathfinding (MAPF) problem seeks collision-free paths for a team of agents from their current positions to their pre-set goals in a known environment, and is an essential problem found at the core of many logistics, transportation, and general robotics applications. Existing learning-based MAPF approaches typically only let each agent make decisions based on a limited field-of-view (FOV) around its position, as a natural means to fix the input dimensions of its policy network. However, this often makes policies short-sighted, since agents lack the ability to perceive and plan for obstacles/agents beyond their FOV. To address this challenge, we propose ALPHA, a new framework combining the use of ground truth proximal (local) information and fuzzy distal (global) information to let agents sequence local decisions based on the full current state of the system, and avoid such myopicity. We further allow agents to make short-term predictions about each others' paths, as a means to reason about each others' path intentions, thereby enhancing the level of cooperation among agents at the whole system level. Our neural structure relies on a Graph Transformer architecture to allow agents to selectively combine these different sources of information and reason about their inter-dependencies at different spatial scales. Our simulation experiments demonstrate that ALPHA outperforms both globally-guided MAPF solvers and communication-learning based ones, showcasing its potential towards scalability in realistic deployments.

Robotics
Compilation-based Solvers for Multi-Agent Path Finding: a Survey, Discussion, and Future Opportunities

Pavel Surynek

DOI: https://doi.org/10.48550/arXiv.2104.11809

2021-04-24

Abstract:Multi-agent path finding (MAPF) attracts considerable attention in artificial intelligence community as well as in robotics, and other fields such as warehouse logistics. The task in the standard MAPF is to find paths through which agents can navigate from their starting positions to specified individual goal positions. The combination of two additional requirements makes the problem computationally challenging: (i) agents must not collide with each other and (ii) the paths must be optimal with respect to some objective. Two major approaches to optimal MAPF solving include (1) dedicated search-based methods, which solve MAPF directly, and (2) compilation-based methods that reduce a MAPF instance to an instance in a different well established formalism, for which an efficient solver exists. The compilation-based MAPF solving can benefit from advancements accumulated during the development of the target solver often decades long. We summarize and compare contemporary compilation-based solvers for MAPF using formalisms like ASP, MIP, and SAT. We show the lessons learned from past developments and current trends in the topic and discuss its wider impact.

Artificial Intelligence,Multiagent Systems
A Review of Graph-Based Multi-Agent Pathfinding Solvers: from Classical to Beyond Classical

Jianqi Gao,Yanjie Li,Xinyi Li,Kejian Yan,Ke Lin,Xinyu Wang

DOI: https://doi.org/10.1016/j.knosys.2023.111121

IF: 8.139

2024-01-01

Knowledge-Based Systems

Abstract:Multi-agent pathfinding (MAPF) is a well-studied abstract model for navigation in a multi-robot system, where every robot finds the path to its goal position without any collision. Due to its numerous practical applications of multi-robot systems, MAPF has steadily emerged as a research hotspot. The optimal solution for MAPF is NP-hard. In this paper, we offer a comprehensive analysis of different MAPF solvers. First, we review the cutting-edge solvers of classical MAPF, including optimal, bounded sub-optimal, and unbounded sub-optimal. The performance of some representative classical MAPF solvers is quantitatively compared. In the next part, we summarize the beyond classical MAPF solvers, which try to use the classical MAPF solvers in real-world scenarios. Last, we conclude some challenges that MAPF is experiencing in detail, review recent research on these issues, and make some suggestions for further work.
Multi-Agent Vulcan: An Information-Driven Multi-Agent Path Finding Approach

Jake Olkin,Viraj Parimi,Brian Williams

2024-09-20

Abstract:Scientists often search for phenomena of interest while exploring new environments. Autonomous vehicles are deployed to explore such areas where human-operated vehicles would be costly or dangerous. Online control of autonomous vehicles for information-gathering is called adaptive sampling and can be framed as a POMDP that uses information gain as its principal objective. While prior work focuses largely on single-agent scenarios, this paper confronts challenges unique to multi-agent adaptive sampling, such as avoiding redundant observations, preventing vehicle collision, and facilitating path planning under limited communication. We start with Multi-Agent Path Finding (MAPF) methods, which address collision avoidance by decomposing the MAPF problem into a series of single-agent path planning problems. We then present information-driven MAPF which addresses multi-agent information gain under limited communication. First, we introduce an admissible heuristic that relaxes mutual information gain to an additive function that can be evaluated as a set of independent single agent path planning problems. Second, we extend our approach to a distributed system that is robust to limited communication. When all agents are in range, the group plans jointly to maximize information. When some agents move out of range, communicating subgroups are formed and the subgroups plan independently. Since redundant observations are less likely when vehicles are far apart, this approach only incurs a small loss in information gain, resulting in an approach that gracefully transitions from full to partial communication. We evaluate our method against other adaptive sampling strategies across various scenarios, including real-world robotic applications. Our method was able to locate up to 200% more unique phenomena in certain scenarios, and each agent located its first unique phenomenon faster by up to 50%.

Multiagent Systems,Robotics
Multi-agent Pathfinding with Local and Global Guidance

Yunhong Xu,Yanjie Li,Qi Liu,Jianqi Gao,Yuecheng Liu,Meiling Chen

DOI: https://doi.org/10.1109/icnsc52481.2021.9702234

2021-01-01

Abstract:Multi-agent path finding (MAPF) exists in many practical applications, such as intelligent warehouses. In this type of scenario, the agents need to cooperate with each other and eventually reach the target point without collision. The existing multi-agent path planning algorithms are mainly centralized algorithms, such as Conflict-Based Search (CBS). However, this kind of approach is difficult to solve the problem in real time and its scalability is poor. In this work, we focus on solving MAPF problem in intelligent warehouse. To address this problem, instead of using time-consuming search based algorithms, we propose a novel decentralized multi-agent pathfinding method based on deep reinforcement learning. Combined with curriculum learning, the algorithm uses local and global guidance mechanisms to help agents plan feasible paths. As a result, the success rate of the algorithm has been significantly improved. Experimental results show that our algorithm generalizes well and it still performs well when the scale of problem increases. The solution efficiency is close to the centralized algorithms.
Multi-Agent Path Finding with Delay Probabilities

Hang Ma,T. K. Satish Kumar,Sven Koenig

DOI: https://doi.org/10.1609/aaai.v31i1.11035

2017-02-12

Proceedings of the AAAI Conference on Artificial Intelligence

Abstract:Several recently developed Multi-Agent Path Finding (MAPF) solvers scale to large MAPF instances by searching for MAPF plans on 2 levels: The high-level search resolves collisions between agents, and the low-level search plans paths for single agents under the constraints imposed by the high-level search. We make the following contributions to solve the MAPF problem with imperfect plan execution with small average makespans: First, we formalize the MAPF Problem with Delay Probabilities (MAPF-DP), define valid MAPF-DP plans and propose the use of robust plan-execution policies for valid MAPF-DP plans to control how each agent proceeds along its path. Second, we discuss 2 classes of decentralized robust plan-execution policies (called Fully Synchronized Policies and Minimal Communication Policies) that prevent collisions during plan execution for valid MAPF-DP plans. Third, we present a 2-level MAPF-DP solver (called Approximate Minimization in Expectation) that generates valid MAPF-DP plans.

English Else
Towards Deployment-Efficient and Collision-Free Multi-Agent Path Finding (student Abstract).

Feng Chen,Chenghe Wang,Fuxiang Zhang,Hao Ding,Qiaoyong Zhong,Shiliang Pu,Zongzhang Zhang

DOI: https://doi.org/10.1609/aaai.v37i13.26951

2023-01-01

Abstract:Multi-agent pathfinding (MAPF) is essential to large-scale robotic coordination tasks. Planning-based algorithms show their advantages in collision avoidance while avoiding exponential growth in the number of agents. Reinforcement-learning (RL)-based algorithms can be deployed efficiently but cannot prevent collisions entirely due to the lack of hard constraints. This paper combines the merits of planning-based and RL-based MAPF methods to propose a deployment-efficient and collision-free MAPF algorithm. The experiments show the effectiveness of our approach.
Multi-Agent Path Finding Method Based on Evolutionary Reinforcement Learning

Qinru Shi,Meiqin Liu,Senlin Zhang,Ronghao Zheng,Xuguang Lan

DOI: https://doi.org/10.23919/ccc63176.2024.10661475

2024-01-01

Abstract:The multi-agent path finding (MAPF) problem is crucial to improve the efficiency of warehouse systems. Compared with traditional centralized methods, which encounter escalating computational complexities with increasing scale, reinforcement learning-based methods has been proven to be an effective method for solving MAPF problem. Nevertheless, in the complex and large-scale scenarios, the policies learned by existing reinforcement learning-based methods are generally inadequate to address the challenges effectively. By leveraging the concepts of policy evaluation and policy evolution, this paper aims to improve performance and sample efficiency. Consequently, we introduce an MAPF method based on evolutionary reinforcement learning. In particular, we design a collaborative policy network model based on reinforcement learning. Thereafter, a novel evolutionary reinforcement learning training framework is constructed. Through the quantitative evaluation mechanism, policy evaluation is carried out, and evolutionary algorithm is used for policy evolution, so that the collaborative policy could better guide the agent to complete the path finding task. We test on high-density warehouse environment instances of various map sizes, and the experimental results show that our method has high success rate and low average steps.
Caching-Augmented Lifelong Multi-Agent Path Finding

Yimin Tang,Zhenghong Yu,Yi Zheng,T. K. Satish Kumar,Jiaoyang Li,Sven Koenig

2024-04-06

Abstract:Multi-Agent Path Finding (MAPF), which involves finding collision-free paths for multiple robots, is crucial in various applications. Lifelong MAPF, where targets are reassigned to agents as soon as they complete their initial targets, offers a more accurate approximation of real-world warehouse planning. In this paper, we present a novel mechanism named Caching-Augmented Lifelong MAPF (CAL-MAPF), designed to improve the performance of Lifelong MAPF. We have developed a new type of map grid called cache for temporary item storage and replacement, and created a locking mechanism to improve the planning solution's stability. A task assigner (TA) is designed for CAL-MAPF to allocate target locations to agents and control agent status in different situations. CAL-MAPF has been evaluated using various cache replacement policies and input task distributions. We have identified three main factors significantly impacting CAL-MAPF performance through experimentation: suitable input task distribution, high cache hit rate, and smooth traffic. In general, CAL-MAPF has demonstrated potential for performance improvements in certain task distributions, map and agent configurations.

Robotics,Artificial Intelligence,Multiagent Systems
The Multi-Agent Pickup and Delivery Problem: MAPF, MARL and Its Warehouse Applications

Tim Tsz-Kit Lau,Biswa Sengupta

DOI: https://doi.org/10.48550/arXiv.2203.07092

2022-03-14

Abstract:We study two state-of-the-art solutions to the multi-agent pickup and delivery (MAPD) problem based on different principles -- multi-agent path-finding (MAPF) and multi-agent reinforcement learning (MARL). Specifically, a recent MAPF algorithm called conflict-based search (CBS) and a current MARL algorithm called shared experience actor-critic (SEAC) are studied. While the performance of these algorithms is measured using quite different metrics in their separate lines of work, we aim to benchmark these two methods comprehensively in a simulated warehouse automation environment.

Machine Learning,Multiagent Systems
Multi Agent Pathfinding for Noise Restricted Hybrid Fuel Unmanned Aerial Vehicles

Drew Scott,Satyanarayana G. Manyam,David W. Casbeer,Manish Kumar,Isaac E. Weintraub

2024-03-27

Abstract:Multi Agent Path Finding (MAPF) seeks the optimal set of paths for multiple agents from respective start to goal locations such that no paths conflict. We address the MAPF problem for a fleet of hybrid-fuel unmanned aerial vehicles which are subject to location-dependent noise restrictions. We solve this problem by searching a constraint tree for which the subproblem at each node is a set of shortest path problems subject to the noise and fuel constraints and conflict zone avoidance. A labeling algorithm is presented to solve this subproblem, including the conflict zones which are treated as dynamic obstacles. We present the experimental results of the algorithms for various graph sizes and number of agents.

Optimization and Control,Robotics
A Comprehensive Review on Leveraging Machine Learning for Multi-Agent Path Finding

Jean-Marc Alkazzi,Keisuke Okumura

DOI: https://doi.org/10.1109/access.2024.3392305

IF: 3.9

2024-04-30

IEEE Access

Abstract:This review paper provides an in-depth analysis of the latest advancements in applying Machine Learning (ML) to solve the Multi-Agent Path Finding (MAPF) problem. The MAPF problem is about finding collision-free paths for multiple agents to travel from their source to goal locations in a known environment. This method underpins a range of advanced, large-scale automated systems, notably in warehouse logistics. The existing research on conventional MAPF is extensive; however, recent developments in ML have notably augmented the capabilities of MAPF techniques. This research seeks to thoroughly investigate the emerging field focused on using ML to help solve the MAPF problem. It aims to highlight the transformative potential of ML in enhancing the efficiency and effectiveness of multi-agent systems in navigating and coordinating in complex environments. Our study comprehensively examines the entire MAPF process, encompassing environment representation, path planning, and solution execution.

computer science, information systems,telecommunications,engineering, electrical & electronic
Multi-Agent Path Finding Based on Subdimensional Expansion with Bypass

Qingzhou Liu,Feng Wu

DOI: https://doi.org/10.48550/arXiv.2207.14657

2022-07-29

Abstract:Multi-agent path finding (MAPF) is an active area in artificial intelligence, which has many real-world applications such as warehouse management, traffic control, robotics, etc. Recently, M* and its variants have greatly improved the ability to solve the MAPF problem. Although subdimensional expansion used in those approaches significantly decreases the dimensionality of the joint search space and reduces the branching factor, they do not make full use of the possible non-uniqueness of the optimal path of each agent. As a result, the updating of the collision sets may bring a large number of redundant computation. In this paper, the idea of bypass is introduced into subdimensional expansion to reduce the redundant computation. Specifically, we propose the BPM* algorithm, which is an implementation of subdimensional expansion with bypass in M*. In the experiments, we show that BPM* outperforms the state-of-the-art in solving several MAPF benchmark problems.

Robotics
Optimal and Bounded Suboptimal Any-Angle Multi-agent Pathfinding

Konstantin Yakovlev,Anton Andreychuk,Roni Stern

2024-08-30

Abstract:Multi-agent pathfinding (MAPF) is the problem of finding a set of conflict-free paths for a set of agents. Typically, the agents' moves are limited to a pre-defined graph of possible locations and allowed transitions between them, e.g. a 4-neighborhood grid. We explore how to solve MAPF problems when each agent can move between any pair of possible locations as long as traversing the line segment connecting them does not lead to a collision with the obstacles. This is known as any-angle pathfinding. We present the first optimal any-angle multi-agent pathfinding algorithm. Our planner is based on the Continuous Conflict-based Search (CCBS) algorithm and an optimal any-angle variant of the Safe Interval Path Planning (TO-AA-SIPP). The straightforward combination of those, however, scales poorly since any-angle path finding induces search trees with a very large branching factor. To mitigate this, we adapt two techniques from classical MAPF to the any-angle setting, namely Disjoint Splitting and Multi-Constraints. Experimental results on different combinations of these techniques show they enable solving over 30% more problems than the vanilla combination of CCBS and TO-AA-SIPP. In addition, we present a bounded-suboptimal variant of our algorithm, that enables trading runtime for solution cost in a controlled manner.

Artificial Intelligence,Multiagent Systems,Robotics
Lifelong Multi-Agent Path Finding for Online Pickup and Delivery Tasks

Hang Ma,Jiaoyang Li,T. K. Satish Kumar,Sven Koenig

DOI: https://doi.org/10.48550/arXiv.1705.10868

2017-05-31

Abstract:The multi-agent path-finding (MAPF) problem has recently received a lot of attention. However, it does not capture important characteristics of many real-world domains, such as automated warehouses, where agents are constantly engaged with new tasks. In this paper, we therefore study a lifelong version of the MAPF problem, called the multi-agent pickup and delivery (MAPD) problem. In the MAPD problem, agents have to attend to a stream of delivery tasks in an online setting. One agent has to be assigned to each delivery task. This agent has to first move to a given pickup location and then to a given delivery location while avoiding collisions with other agents. We present two decoupled MAPD algorithms, Token Passing (TP) and Token Passing with Task Swaps (TPTS). Theoretically, we show that they solve all well-formed MAPD instances, a realistic subclass of MAPD instances. Experimentally, we compare them against a centralized strawman MAPD algorithm without this guarantee in a simulated warehouse system. TP can easily be extended to a fully distributed MAPD algorithm and is the best choice when real-time computation is of primary concern since it remains efficient for MAPD instances with hundreds of agents and tasks. TPTS requires limited communication among agents and balances well between TP and the centralized MAPD algorithm.

Artificial Intelligence,Multiagent Systems,Robotics

AI Buzzwords Explained: Multi-Agent Path Finding (MAPF)

Overview: Generalizations of Multi-Agent Path Finding to Real-World Scenarios

Multi-Agent Path Finding with heterogeneous edges and roundtrips

Learning to Cooperate: Application of Deep Reinforcement Learning for Online AGV Path Finding.

Multi-Agent Path Finding with Heterogeneous Geometric and Kinematic Constraints in Continuous Space

MAPF and MAPD: Recent Developments and Future Directions (short paper)

ALPHA: Attention-based Long-horizon Pathfinding in Highly-structured Areas

Compilation-based Solvers for Multi-Agent Path Finding: a Survey, Discussion, and Future Opportunities

A Review of Graph-Based Multi-Agent Pathfinding Solvers: from Classical to Beyond Classical

Multi-Agent Vulcan: An Information-Driven Multi-Agent Path Finding Approach

Multi-agent Pathfinding with Local and Global Guidance

Multi-Agent Path Finding with Delay Probabilities

Towards Deployment-Efficient and Collision-Free Multi-Agent Path Finding (student Abstract).

Multi-Agent Path Finding Method Based on Evolutionary Reinforcement Learning

Caching-Augmented Lifelong Multi-Agent Path Finding

The Multi-Agent Pickup and Delivery Problem: MAPF, MARL and Its Warehouse Applications

Multi Agent Pathfinding for Noise Restricted Hybrid Fuel Unmanned Aerial Vehicles

A Comprehensive Review on Leveraging Machine Learning for Multi-Agent Path Finding

Multi-Agent Path Finding Based on Subdimensional Expansion with Bypass

Optimal and Bounded Suboptimal Any-Angle Multi-agent Pathfinding

Lifelong Multi-Agent Path Finding for Online Pickup and Delivery Tasks