Abstract:Deep reinforcement learning (DRL) has been proven as a powerful approach for robot navigation over the past few years. DRL-based navigation does not require the pre-construction of a map, instead, high-performance navigation skills can be learned from trial-and-error experiences. However, recent DRL-based approaches mostly focus on a fixed navigation target. It is noted that when navigating to a moving target without maps, the performance of the standard RL structure drops dramatically on both the success rate and path efficiency. To address the mapless navigation problem with moving target, the predictive hierarchical DRL (pH-DRL) framework is proposed by integrating the long-term trajectory prediction to provide a cost-effective solution. In the proposed framework, the lower-level policy of the RL agent learns robot control actions to a specified goal, and the higher-level policy learns to make long-range planning of shorter navigation routes by sufficiently exploiting the predicted trajectories. By means of making decisions over two level of policies, the pH-DRL framework is robust to the unavoidable errors in long-term predictions. With the application of deep deterministic policy gradient (DDPG) for policy optimization, the pH-DDPG algorithm is developed based on the pH-DRL structure. Finally, through comparative experiments on the Gazebo simulator with several variants of the DDPG algorithm, the results demonstrate that the pH-DDPG outperforms other algorithms and achieves a high success rate and efficiency even though the target moves fast and randomly.

Probabilistic Autonomous Robot Navigation in Dynamic Environments with Human Motion Prediction

Predictive Hierarchical Reinforcement Learning for Path-Efficient Mapless Navigation with Moving Target.

POMDP Navigation of Service Robots with Human Motion Prediction

Real-Time Navigation In Dynamic Human Environments Using Optimal Reciprocal Collision Avoidance

Probabilistic Roadmap With Self-Learning For Path Planning Of A Mobile Robot In A Dynamic And Unstructured Environment

Towards Using Fast Embedded Model Predictive Control for Human-Aware Predictive Robot Navigation

Predictive Probability Path Planning Model For Dynamic Environments

Map-Predictive Motion Planning in Unknown Environments

A Local Path Planning of Mobile Robots in Dynamic Environment Based on Prediction of Collision

High-Speed Robot Navigation using Predicted Occupancy Maps

Robot Navigation Anticipative Strategies in Deep Reinforcement Motion Planning

Dynamics-aware navigation among moving obstacles with application to ground and flying robots

Anticipatory Navigation in Crowds by Probabilistic Prediction of Pedestrian Future Movements

OA-MPC: Occlusion-Aware MPC for Guaranteed Safe Robot Navigation with Unseen Dynamic Obstacles

Robot navigation with predictive capabilities using graph learning and Monte Carlo tree search

Intent Prediction-Driven Model Predictive Control for UAV Planning and Navigation in Dynamic Environments

Fast Adaptable Mobile Robot Navigation in Dynamic Environment

Robot Navigation Based on Human Trajectory Prediction and Multiple Travel Modes

Probabilistically Safe Robot Planning with Confidence-Based Human Predictions

Improving Reliable Navigation under Uncertainty via Predictions Informed by Non-Local Information

Robust Optimal Navigation Using Nonlinear Model Predictive Control Method Combined with Recurrent Fuzzy Neural Network