Abstract:Compared with a single robot, Multi-robot Systems (MRSs) can undertake more challenging tasks in complex scenarios benefiting from the increased transportation capacity and fault tolerance. This paper presents a hierarchical framework for multi-robot navigation and formation in unknown environments with static and dynamic obstacles, where the robots compute and maintain the optimized formation while making progress to the target together. In the proposed framework, each single robot is capable of navigating to the global target in unknown environments based on its local perception, and only limited communication among robots is required to obtain the optimal formation. Accordingly, three modules are included in this framework. Firstly, we design a learning network based on Deep Deterministic Policy Gradient (DDPG) to address the global navigation task for single robot, which derives end-to-end policies that map the robot's local perception into its velocity commands. To handle complex obstacle distributions (e.g. narrow/zigzag passage and local minimum) and stabilize the training process, strategies of Curriculum Learning (CL) and Reward Shaping (RS) are combined. Secondly, for an expected formation, its real-time configuration is optimized by a distributed optimization. This configuration considers surrounding obstacles and current formation status, and provides each robot with its formation target. Finally, a velocity adjustment method considering the robot kinematics is designed which adjusts the navigation velocity of each robot according to its formation target, making all the robots navigate to their targets while maintaining the expected formation. This framework allows for formation online reconfiguration and is scalable with the number of robots. Extensive simulations and 3-D evaluations verify that our method can navigate the MRS in unknown environments while maintaining the optimal formation.

Multi-robot Cooperation Strategy in a Partially Observable Markov Game Using Enhanced Deep Deterministic Policy Gradient

Mapless Collaborative Navigation for a Multi-Robot System Based on the Deep Reinforcement Learning

Cooperative multi-agent target searching: a deep reinforcement learning approach based on parallel hindsight experience replay

Dueling Network Architecture for Multi-Agent Deep Deterministic Policy Gradient

An Improved Approach Towards Multi-Agent Pursuit–Evasion Game Decision-Making Using Deep Reinforcement Learning

Twin Delayed Multi-Agent Deep Deterministic Policy Gradient

Pursuit-Evasion Games for Multi-agent Based on Reinforcement Learning with Obstacles

Multi-Robot Real-time Game Strategy Learning Based on Deep Reinforcement Learning.

A TD3-based multi-agent deep reinforcement learning method in mixed cooperation-competition environment

A Multi-agent Reinforcement Learning Method for Swarm Robots in Space Collaborative Exploration

Decentralized Multi-Agent Reinforcement Learning with Global State Prediction

Collision-aware Multi-robot Motion Coordination Deep-RL with Dynamic Priority Strategy

Motion Coordination Of Multiple Robots Based On Deep Reinforcement Learning

Multi-Agent Distributed Deep Deterministic Policy Gradient for Partially Observable Tracking

Hierarchical Reinforcement Learning with Opponent Modeling for Distributed Multi-agent Cooperation

Towards Optimally Decentralized Multi-Robot Collision Avoidance Via Deep Reinforcement Learning

Hierarchical Multi-Robot Navigation and Formation in Unknown Environments Via Deep Reinforcement Learning and Distributed Optimization

A Deep Reinforcement Learning-Based Method Applied for Solving Multi-Agent Defense and Attack Problems.

A Collaborative Control Method of Dual-Arm Robots Based on Deep Reinforcement Learning

Reinforcement Learned Distributed Multi-Robot Navigation With Reciprocal Velocity Obstacle Shaped Rewards