Abstract:It is challenging to avoid obstacles safely and efficiently for multiple robots of different shapes in distributed and communication-free scenarios, where robots do not communicate with each other and only sense other robots’ positions and obstacles around them. Most existing multi-robot collision avoidance systems either require communication between robots or require expensive movement data of other robots, like velocities, accelerations and paths. In this paper, we propose a map-based deep reinforcement learning approach for multi-robot collision avoidance in a distributed and communication-free environment. We use the egocentric local grid map of a robot to represent the environmental information around it including its shape and observable appearances of other robots and obstacles, which can be easily generated by using multiple sensors or sensor fusion. Then we apply the distributed proximal policy optimization (DPPO) algorithm to train a convolutional neural network that directly maps three frames of egocentric local grid maps and the robot’s relative local goal positions into low-level robot control commands. Compared to other methods, the map-based approach is more robust to noisy sensor data, does not require robots’ movement data and considers sizes and shapes of related robots, which make it to be more efficient and easier to be deployed to real robots. We first train the neural network in a specified simulator of multiple mobile robots using DPPO, where a multi-stage curriculum learning strategy for multiple scenarios is used to improve the performance. Then we deploy the trained model to real robots to perform collision avoidance in their navigation without tedious parameter tuning. We evaluate the approach with multiple scenarios both in the simulator and on four differential-drive mobile robots in the real world. Both qualitative and quantitative experiments show that our approach is efficient and outperforms existing DRL-based approaches in many indicators. We also conduct ablation studies showing the positive effects of using egocentric grid maps and multi-stage curriculum learning.

Autonomous spacecraft collision avoidance with a variable number of space debris based on safe reinforcement learning

Autonomous Spacecraft Collision Avoidance with Multiple Space Debris Based on Reinforcement Learning

Spacecraft Attitude Maneuver Planning Based on Deep Reinforcement Learning under Complex Constraints

Spacecraft Autonomous Decision-Planning for Collision Avoidance: a Reinforcement Learning Approach

A Reinforcement Learning-based Decentralized Method of Avoiding Multi-UAV Collision in 3-D Airspace

Research on collision avoidance algorithm of unmanned surface vehicle based on deep reinforcement learning

Collision avoidance decision-making strategy for multiple USVs based on Deep Reinforcement Learning algorithm

A Novel Deep Reinforcement Learning for POMDP-based Autonomous Ship Collision Decision-Making

Coordinated Control Based on Reinforcement Learning for Dual-Arm Continuum Manipulators in Space Capture Missions

Revisiting Space Mission Planning: A Reinforcement Learning-Guided Approach for Multi-Debris Rendezvous

Mission Planning on Autonomous Avoidance for Spacecraft Confronting Orbital Debris

Proximal policy optimization with reciprocal velocity obstacle based collision avoidance path planning for multi-unmanned surface vehicles

A partially observable multi-ship collision avoidance decision-making model based on deep reinforcement learning

Multi-UAV simultaneous target assignment and path planning based on deep reinforcement learning in dynamic multiple obstacles environments

Obstacle Avoidance for UAS in Continuous Action Space Using Deep Reinforcement Learning

AI-Driven Risk-Aware Scheduling for Active Debris Removal Missions

A COLREGs-Compliant Deep Reinforcement Learning Approach

Active Debris Removal Mission Planning Method Based on Machine Learning

Collision avoidance for autonomous ship using deep reinforcement learning and prior-knowledge-based approximate representation

COLERGs-constrained Safe Reinforcement Learning for Realising MASS's Risk-Informed Collision Avoidance Decision Making

Distributed Non-Communicating Multi-Robot Collision Avoidance Via Map-Based Deep Reinforcement Learning