Abstract:Navigating mobile robots along time-efficient and collision-free paths in crowds is still an open and challenging problem. The key is to build a profound understanding of the crowd for mobile robots, which is the basis of a proactive and foresighted policy. However, since the interaction mechanisms among pedestrians are complex and sophisticated, it is difficult to describe and model them accurately. For the excellent approximation capability of deep neural networks, deep reinforcement learning is a promising solution to the problem. However, current model-free learning-based approaches in crowd navigation always neglect planning and still lead to reactive collision avoidance policies and shortsighted behaviors. Meanwhile, most model-based learning-based approaches are based on state values, imposing a substantial computational burden. To address these problems, we propose a graph-based deep reinforcement learning method, social graph-based double dueling deep Q-network (SG-D3QN), that (i) introduces a social attention mechanism to extract an efficient graph representation for the crowd-robot state, (ii) extends the previous state value approximator to a state-action value approximator, (iii) further optimizes the state-action value approximator with simulated experiences generated by the learned environment model, and (iv) then proposes a human-like decision-making process by integrating model-free reinforcement learning and online planning. Experimental results indicate that our approach helps the robot understand the crowd and achieves a high success rate of more than 0.99 in the crowd navigation task. Compared with previous state-of-the-art algorithms, our approach achieves better performance. Furthermore, with the human-like decision-making process, our approach incurs less than half of the computational cost.

Spatio-Temporal Transformer-Based Reinforcement Learning for Robot Crowd Navigation

ST$^{2}$: Spatial-Temporal State Transformer for Crowd-Aware Autonomous Navigation

Learning World Transition Model for Socially Aware Robot Navigation

Robot Navigation Using Reinforcement Learning with Multi Attention Fusion in Crowd

Crowd-Aware Socially Compliant Robot Navigation via Deep Reinforcement Learning

Robot navigation in a crowd by integrating deep reinforcement learning and online planning

Multi-objective deep reinforcement learning for crowd-aware robot navigation with dynamic human preference

Intention Aware Robot Crowd Navigation with Attention-Based Interaction Graph

Modeling Spatial-Temporal Interactions for Robot Crowd Navigation

Robot Mapless Navigation in VUCA Environments Via Deep Reinforcement Learning

Robot Crowd Navigation in Dynamic Environment with Offline Reinforcement Learning

NaviSTAR: Socially Aware Robot Navigation with Hybrid Spatio-Temporal Graph Transformer and Preference Learning

Transformer-Based Reinforcement Learning for Multi-Robot Autonomous Exploration

Multi-Robot Cooperative Socially-Aware Navigation Using Multi-Agent Reinforcement Learning

Control Transformer: Robot Navigation in Unknown Environments through PRM-Guided Return-Conditioned Sequence Modeling

Robot Crowd Navigation Based on Spatio-Temporal Interaction Graphs and Danger Zones

Robot Navigation in Crowd Based on Dual Social Attention Deep Reinforcement Learning

Robot navigation in crowds via deep reinforcement learning with modeling of obstacle uni-action

Learning to Socially Navigate in Pedestrian-rich Environments with Interaction Capacity

Learning Crowd Behaviors in Navigation with Attention-based Spatial-Temporal Graphs

Deep reinforcement learning navigation via decision transformer in autonomous driving