Abstract:It is challenging to avoid obstacles safely and efficiently for multiple robots of different shapes in distributed and communication-free scenarios, where robots do not communicate with each other and only sense other robots’ positions and obstacles around them. Most existing multi-robot collision avoidance systems either require communication between robots or require expensive movement data of other robots, like velocities, accelerations and paths. In this paper, we propose a map-based deep reinforcement learning approach for multi-robot collision avoidance in a distributed and communication-free environment. We use the egocentric local grid map of a robot to represent the environmental information around it including its shape and observable appearances of other robots and obstacles, which can be easily generated by using multiple sensors or sensor fusion. Then we apply the distributed proximal policy optimization (DPPO) algorithm to train a convolutional neural network that directly maps three frames of egocentric local grid maps and the robot’s relative local goal positions into low-level robot control commands. Compared to other methods, the map-based approach is more robust to noisy sensor data, does not require robots’ movement data and considers sizes and shapes of related robots, which make it to be more efficient and easier to be deployed to real robots. We first train the neural network in a specified simulator of multiple mobile robots using DPPO, where a multi-stage curriculum learning strategy for multiple scenarios is used to improve the performance. Then we deploy the trained model to real robots to perform collision avoidance in their navigation without tedious parameter tuning. We evaluate the approach with multiple scenarios both in the simulator and on four differential-drive mobile robots in the real world. Both qualitative and quantitative experiments show that our approach is efficient and outperforms existing DRL-based approaches in many indicators. We also conduct ablation studies showing the positive effects of using egocentric grid maps and multi-stage curriculum learning.

Multirobot Unknown Environment Exploration and Obstacle Avoidance Based on a Voronoi Diagram and Reinforcement Learning

Multi-Robot Learning Dynamic Obstacle Avoidance in Formation with Information-Directed Exploration.

Mapless Collaborative Navigation for a Multi-Robot System Based on the Deep Reinforcement Learning

Efficient Autonomous Exploration of Unknown Environment Using Regions Segmentation and VRP

Formation Rotation and Assignment: Avoiding Obstacles in Multi-Robot Scenarios

Hierarchical Multi-Robot Navigation and Formation in Unknown Environments Via Deep Reinforcement Learning and Distributed Optimization

HMS-RRT: A novel hybrid multi-strategy rapidly-exploring random tree algorithm for multi-robot collaborative exploration in unknown environments

Multi-robot collaborative hunting in cluttered environments with obstacle-avoiding voronoi cells

Voronoi-based Multi-robot Collision Avoidance with Localization Uncertainty

A Combined Hierarchical Reinforcement Learning Based Approach for Multi-Robot Cooperative Target Searching in Complex Unknown Environments

Multi-Robot Environmental Coverage With a Two-Stage Coordination Strategy via Deep Reinforcement Learning

Autonomous Multirobot Navigation and Cooperative Mapping in Partially Unknown Environments

Bio-inspired Decentralized Multi-robot Exploration in Unknown Environments

Distributed Non-Communicating Multi-Robot Collision Avoidance Via Map-Based Deep Reinforcement Learning

Resilient and Adaptive Replanning for Multi-Robot Target Tracking with Sensing and Communication Danger Zones

Formation and Obstacle Avoidance in the Unknown Environment of Multi-Robot System

Obstacle Avoidance Algorithm for Mobile Robot Based on Deep Reinforcement Learning in Dynamic Environments.

Multirobot Coordination with Deep Reinforcement Learning in Complex Environments.

An Enhanced Hierarchical Planning Framework for Multi-Robot Autonomous Exploration

Multi-Robot Autonomous Exploration in Unknown Environment: A Review

Multi-robots Formation and Navigation Based Reinforcement Learning