Abstract:Compared with a single robot, Multi-robot Systems (MRSs) can undertake more challenging tasks in complex scenarios benefiting from the increased transportation capacity and fault tolerance. This paper presents a hierarchical framework for multi-robot navigation and formation in unknown environments with static and dynamic obstacles, where the robots compute and maintain the optimized formation while making progress to the target together. In the proposed framework, each single robot is capable of navigating to the global target in unknown environments based on its local perception, and only limited communication among robots is required to obtain the optimal formation. Accordingly, three modules are included in this framework. Firstly, we design a learning network based on Deep Deterministic Policy Gradient (DDPG) to address the global navigation task for single robot, which derives end-to-end policies that map the robot's local perception into its velocity commands. To handle complex obstacle distributions (e.g. narrow/zigzag passage and local minimum) and stabilize the training process, strategies of Curriculum Learning (CL) and Reward Shaping (RS) are combined. Secondly, for an expected formation, its real-time configuration is optimized by a distributed optimization. This configuration considers surrounding obstacles and current formation status, and provides each robot with its formation target. Finally, a velocity adjustment method considering the robot kinematics is designed which adjusts the navigation velocity of each robot according to its formation target, making all the robots navigate to their targets while maintaining the expected formation. This framework allows for formation online reconfiguration and is scalable with the number of robots. Extensive simulations and 3-D evaluations verify that our method can navigate the MRS in unknown environments while maintaining the optimal formation.

Moving Forward in Formation: A Decentralized Hierarchical Learning Approach to Multi-Agent Moving Together

Learning to Cooperate: Application of Deep Reinforcement Learning for Online AGV Path Finding.

Mapless Collaborative Navigation for a Multi-Robot System Based on the Deep Reinforcement Learning

Multi-agent Pathfinding with Local and Global Guidance

Relative Distributed Formation and Obstacle Avoidance with Multi-agent Reinforcement Learning

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

Formation Tracking of Spatiotemporal Multiagent Systems: A Decentralized Reinforcement Learning Approach

Multi-Agent Path Finding Method Based on Evolutionary Reinforcement Learning

Learn to Follow: Decentralized Lifelong Multi-agent Pathfinding via Planning and Learning

ALPHA: Attention-based Long-horizon Pathfinding in Highly-structured Areas

Reinforcement learning for multi-agent formation navigation with scalability

HELSA: Hierarchical Reinforcement Learning with Spatiotemporal Abstraction for Large-Scale Multi-Agent Path Finding

Hierarchical Multi-Robot Navigation and Formation in Unknown Environments Via Deep Reinforcement Learning and Distributed Optimization

Multi-Agent Path Finding via Reinforcement Learning with Hybrid Reward

Obstacle Avoidance in Multi-Agent Formation Process Based on Deep Reinforcement Learning

Multi-agent Pathfinding with Communication Reinforcement Learning and Deadlock Detection

Cooperative Reward Shaping for Multi-Agent Pathfinding

MAP3F: a decentralized approach to multi-agent pathfinding and collision avoidance with scalable 1D, 2D, and 3D feature fusion

Flexible Formation Control Using Hausdorff Distance: A Multi-agent Reinforcement Learning Approach.

Attention-Cooperated Reinforcement Learning for Multi-agent Path Planning

MFC-EQ: Mean-Field Control with Envelope Q-Learning for Moving Decentralized Agents in Formation