Abstract:Compared with a single robot, Multi-robot Systems (MRSs) can undertake more challenging tasks in complex scenarios benefiting from the increased transportation capacity and fault tolerance. This paper presents a hierarchical framework for multi-robot navigation and formation in unknown environments with static and dynamic obstacles, where the robots compute and maintain the optimized formation while making progress to the target together. In the proposed framework, each single robot is capable of navigating to the global target in unknown environments based on its local perception, and only limited communication among robots is required to obtain the optimal formation. Accordingly, three modules are included in this framework. Firstly, we design a learning network based on Deep Deterministic Policy Gradient (DDPG) to address the global navigation task for single robot, which derives end-to-end policies that map the robot's local perception into its velocity commands. To handle complex obstacle distributions (e.g. narrow/zigzag passage and local minimum) and stabilize the training process, strategies of Curriculum Learning (CL) and Reward Shaping (RS) are combined. Secondly, for an expected formation, its real-time configuration is optimized by a distributed optimization. This configuration considers surrounding obstacles and current formation status, and provides each robot with its formation target. Finally, a velocity adjustment method considering the robot kinematics is designed which adjusts the navigation velocity of each robot according to its formation target, making all the robots navigate to their targets while maintaining the expected formation. This framework allows for formation online reconfiguration and is scalable with the number of robots. Extensive simulations and 3-D evaluations verify that our method can navigate the MRS in unknown environments while maintaining the optimal formation.

Modular Hierarchical Reinforcement Learning for Multi-Destination Navigation in Hybrid Crowds

Predictive Hierarchical Reinforcement Learning for Path-Efficient Mapless Navigation with Moving Target.

CrowdMove: Autonomous Mapless Navigation in Crowded Scenarios

Multi-Objective Crowd-Aware Robot Navigation System Using Deep Reinforcement Learning

Hierarchical reinforcement learning for handling sparse rewards in multi-goal navigation

Multi-objective deep reinforcement learning for crowd-aware robot navigation with dynamic human preference

HeR-DRL:Heterogeneous Relational Deep Reinforcement Learning for Decentralized Multi-Robot Crowd Navigation

Robot navigation in crowds via deep reinforcement learning with modeling of obstacle uni-action

Hierarchical Reinforcement Learning Framework towards Multi-agent Navigation

Learning Hierarchical Behavior and Motion Planning for Autonomous Driving.

Robot Subgoal-guided Navigation in Dynamic Crowded Environments with Hierarchical Deep Reinforcement Learning

Learning to Navigate Through Complex Dynamic Environment with Modular Deep Reinforcement Learning

Deep Reinforcement Learning for Group-Aware Robot Navigation in Crowds.

A Hierarchical Approach for Mobile Robot Exploration in Pedestrian Crowd

Deep Reinforcement Learning-Based Mapless Crowd Navigation with Perceived Risk of the Moving Crowd for Mobile Robots

Hierarchical Multi-Robot Navigation and Formation in Unknown Environments Via Deep Reinforcement Learning and Distributed Optimization

Mobile Robot Navigation Control Based on Hierarchical Reinforcement Learning

Interactive Robot Navigation in Crowded Pedestrian Environments Based on Deep Reinforcement Learning

Hierarchical and Stable Multiagent Reinforcement Learning for Cooperative Navigation Control

Memory-based crowd-aware robot navigation using deep reinforcement learning

Robot Navigation in Crowd Based on Dual Social Attention Deep Reinforcement Learning