Abstract:Developing a safe and efficient collision-avoidance policy for multiple robots is challenging in the decentralized scenarios where each robot generates its paths with limited observation of other robots’ states and intentions. Prior distributed multi-robot collision-avoidance systems often require frequent inter-robot communication or agent-level features to plan a local collision-free action, which is not robust and computationally prohibitive. In addition, the performance of these methods is not comparable with their centralized counterparts in practice. In this article, we present a decentralized sensor-level collision-avoidance policy for multi-robot systems, which shows promising results in practical applications. In particular, our policy directly maps raw sensor measurements to an agent’s steering commands in terms of the movement velocity. As a first step toward reducing the performance gap between decentralized and centralized methods, we present a multi-scenario multi-stage training framework to learn an optimal policy. The policy is trained over a large number of robots in rich, complex environments simultaneously using a policy-gradient-based reinforcement-learning algorithm. The learning algorithm is also integrated into a hybrid control framework to further improve the policy’s robustness and effectiveness. We validate the learned sensor-level collision-3avoidance policy in a variety of simulated and real-world scenarios with thorough performance evaluations for large-scale multi-robot systems. The generalization of the learned policy is verified in a set of unseen scenarios including the navigation of a group of heterogeneous robots and a large-scale scenario with 100 robots. Although the policy is trained using simulation data only, we have successfully deployed it on physical robots with shapes and dynamics characteristics that are different from the simulated agents, in order to demonstrate the controller’s robustness against the simulation-to-real modeling error. Finally, we show that the collision-avoidance policy learned from multi-robot navigation tasks provides an excellent solution for safe and effective autonomous navigation for a single robot working in a dense real human crowd. Our learned policy enables a robot to make effective progress in a crowd without getting stuck. More importantly, the policy has been successfully deployed on different types of physical robot platforms without tedious parameter tuning. Videos are available at https://sites.google.com/view/hybridmrca.

A safe reinforcement learning approach for autonomous navigation of mobile robots in dynamic environments

Learning Observation-Based Certifiable Safe Policy for Decentralized Multi-Robot Navigation

Safe and socially compliant robot navigation in crowds with fast-moving pedestrians via deep reinforcement learning

Real-Time Navigation In Dynamic Human Environments Using Optimal Reciprocal Collision Avoidance

A Deep Safe Reinforcement Learning Approach for Mapless Navigation.

SSRL: A Safe and Smooth Reinforcement Learning Approach for Collision Avoidance in Navigation

Risk-Aware Deep Reinforcement Learning for Robot Crowd Navigation

Obstacle Avoidance Algorithm for Mobile Robot Based on Deep Reinforcement Learning in Dynamic Environments.

A Dynamic Safety Shield for Safe and Efficient Reinforcement Learning of Navigation Tasks

Reinforcement Learning For Mobile Robot Obstacle Avoidance Under Dynamic Environments

Adaptive Environment Modeling Based Reinforcement Learning for Collision Avoidance in Complex Scenes

Robot obstacle avoidance system using deep reinforcement learning

Distributed Multi-Robot Collision Avoidance Via Deep Reinforcement Learning for Navigation in Complex Scenarios

Dynamic warning zone and a short-distance goal for autonomous robot navigation using deep reinforcement learning

Addressing unpredictable movements of dynamic obstacles with deep reinforcement learning to ensure safe navigation for omni-wheeled mobile robot

Fully Distributed Multi-Robot Collision Avoidance via Deep Reinforcement Learning for Safe and Efficient Navigation in Complex Scenarios

SafeCrowdNav: safety evaluation of robot crowd navigation in complex scenes

Deep Reinforcement Learning of Map-Based Obstacle Avoidance for Mobile Robot Navigation

A 3D Simulation Environment and Navigation Approach for Robot Navigation Via Deep Reinforcement Learning in Dense Pedestrian Environment.

Crowd-Aware Socially Compliant Robot Navigation via Deep Reinforcement Learning

Crowd-Aware Robot Navigation for Pedestrians with Multiple Collision Avoidance Strategies via Map-based Deep Reinforcement Learning