Abstract:Navigating mobile robots along time-efficient and collision-free paths in crowds is still an open and challenging problem. The key is to build a profound understanding of the crowd for mobile robots, which is the basis of a proactive and foresighted policy. However, since the interaction mechanisms among pedestrians are complex and sophisticated, it is difficult to describe and model them accurately. For the excellent approximation capability of deep neural networks, deep reinforcement learning is a promising solution to the problem. However, current model-free learning-based approaches in crowd navigation always neglect planning and still lead to reactive collision avoidance policies and shortsighted behaviors. Meanwhile, most model-based learning-based approaches are based on state values, imposing a substantial computational burden. To address these problems, we propose a graph-based deep reinforcement learning method, social graph-based double dueling deep Q-network (SG-D3QN), that (i) introduces a social attention mechanism to extract an efficient graph representation for the crowd-robot state, (ii) extends the previous state value approximator to a state-action value approximator, (iii) further optimizes the state-action value approximator with simulated experiences generated by the learned environment model, and (iv) then proposes a human-like decision-making process by integrating model-free reinforcement learning and online planning. Experimental results indicate that our approach helps the robot understand the crowd and achieves a high success rate of more than 0.99 in the crowd navigation task. Compared with previous state-of-the-art algorithms, our approach achieves better performance. Furthermore, with the human-like decision-making process, our approach incurs less than half of the computational cost.

Deep Reinforcement Learning Based on Social Spatial–Temporal Graph Convolution Network for Crowd Navigation

Learning Crowd Behaviors in Navigation with Attention-based Spatial-Temporal Graphs

Robot navigation in a crowd by integrating deep reinforcement learning and online planning

Modeling Spatial-Temporal Interactions for Robot Crowd Navigation

Graph Relational Reinforcement Learning for Mobile Robot Navigation in Large-Scale Crowded Environments

Crowd-Aware Mobile Robot Navigation Based on Improved Decentralized Structured RNN via Deep Reinforcement Learning

Attention Graph for Multi-Robot Social Navigation with Deep Reinforcement Learning

Relational Navigation Learning in Continuous Action Space among Crowds

Interaction-Aware Crowd Navigation via Augmented Relational Graph Learning

Robot Navigation in Crowd via DeepReinforcement Learning

Robot Navigation in Crowd Based on Dual Social Attention Deep Reinforcement Learning

Robot Navigation among External Autonomous Agents Through Deep Reinforcement Learning Using Graph Attention Network

Robot Crowd Navigation Based on Spatio-Temporal Interaction Graphs and Danger Zones

Robot navigation with predictive capabilities using graph learning and Monte Carlo tree search

Intention Aware Robot Crowd Navigation with Attention-Based Interaction Graph

Crowd-Aware Socially Compliant Robot Navigation via Deep Reinforcement Learning

Socially-Aware Graph Convolutional Network for Human Trajectory Prediction

Robot Subgoal-guided Navigation in Dynamic Crowded Environments with Hierarchical Deep Reinforcement Learning

Reinforcement Learned Multi–Agent Cooperative Navigation in Hybrid Environment with Relational Graph Learning

Structured Graph Network for Constrained Robot Crowd Navigation with Low Fidelity Simulation

Robot navigation in crowds via deep reinforcement learning with modeling of obstacle uni-action