Abstract:Navigating mobile robots along time-efficient and collision-free paths in crowds is still an open and challenging problem. The key is to build a profound understanding of the crowd for mobile robots, which is the basis of a proactive and foresighted policy. However, since the interaction mechanisms among pedestrians are complex and sophisticated, it is difficult to describe and model them accurately. For the excellent approximation capability of deep neural networks, deep reinforcement learning is a promising solution to the problem. However, current model-free learning-based approaches in crowd navigation always neglect planning and still lead to reactive collision avoidance policies and shortsighted behaviors. Meanwhile, most model-based learning-based approaches are based on state values, imposing a substantial computational burden. To address these problems, we propose a graph-based deep reinforcement learning method, social graph-based double dueling deep Q-network (SG-D3QN), that (i) introduces a social attention mechanism to extract an efficient graph representation for the crowd-robot state, (ii) extends the previous state value approximator to a state-action value approximator, (iii) further optimizes the state-action value approximator with simulated experiences generated by the learned environment model, and (iv) then proposes a human-like decision-making process by integrating model-free reinforcement learning and online planning. Experimental results indicate that our approach helps the robot understand the crowd and achieves a high success rate of more than 0.99 in the crowd navigation task. Compared with previous state-of-the-art algorithms, our approach achieves better performance. Furthermore, with the human-like decision-making process, our approach incurs less than half of the computational cost.

Interaction-Aware Crowd Navigation via Augmented Relational Graph Learning

Learning Hierarchical Graph-Based Policy for Goal-Reaching in Unknown Environments

Intention Aware Robot Crowd Navigation with Attention-Based Interaction Graph

Learning Crowd Behaviors in Navigation with Attention-based Spatial-Temporal Graphs

Mapless Collaborative Navigation for a Multi-Robot System Based on the Deep Reinforcement Learning

CrowdMove: Autonomous Mapless Navigation in Crowded Scenarios

Graph Relational Reinforcement Learning for Mobile Robot Navigation in Large-Scale Crowded Environments

Deep Reinforcement Learning Based on Social Spatial–Temporal Graph Convolution Network for Crowd Navigation

Robot navigation in a crowd by integrating deep reinforcement learning and online planning

Crowd-Aware Socially Compliant Robot Navigation via Deep Reinforcement Learning

Attention Graph for Multi-Robot Social Navigation with Deep Reinforcement Learning

Enhancing Navigational Safety in Crowded Environments using Semantic-Deep-Reinforcement-Learning-based Navigation

Learning to Socially Navigate in Pedestrian-rich Environments with Interaction Capacity

Robot navigation in crowds via deep reinforcement learning with modeling of obstacle uni-action

HeR-DRL:Heterogeneous Relational Deep Reinforcement Learning for Decentralized Multi-Robot Crowd Navigation

Crowd-Aware Mobile Robot Navigation Based on Improved Decentralized Structured RNN via Deep Reinforcement Learning

Robot Navigation in Crowd Based on Dual Social Attention Deep Reinforcement Learning

Memory-based crowd-aware robot navigation using deep reinforcement learning

Robot Navigation with Interaction-based Deep Reinforcement Learning

Robot Subgoal-guided Navigation in Dynamic Crowded Environments with Hierarchical Deep Reinforcement Learning

R-SARL: Crowd-aware Navigation Based Deep Reinforcement Learning for Nonholonomic Robot in Complex Environments