Abstract:Navigating mobile robots along time-efficient and collision-free paths in crowds is still an open and challenging problem. The key is to build a profound understanding of the crowd for mobile robots, which is the basis of a proactive and foresighted policy. However, since the interaction mechanisms among pedestrians are complex and sophisticated, it is difficult to describe and model them accurately. For the excellent approximation capability of deep neural networks, deep reinforcement learning is a promising solution to the problem. However, current model-free learning-based approaches in crowd navigation always neglect planning and still lead to reactive collision avoidance policies and shortsighted behaviors. Meanwhile, most model-based learning-based approaches are based on state values, imposing a substantial computational burden. To address these problems, we propose a graph-based deep reinforcement learning method, social graph-based double dueling deep Q-network (SG-D3QN), that (i) introduces a social attention mechanism to extract an efficient graph representation for the crowd-robot state, (ii) extends the previous state value approximator to a state-action value approximator, (iii) further optimizes the state-action value approximator with simulated experiences generated by the learned environment model, and (iv) then proposes a human-like decision-making process by integrating model-free reinforcement learning and online planning. Experimental results indicate that our approach helps the robot understand the crowd and achieves a high success rate of more than 0.99 in the crowd navigation task. Compared with previous state-of-the-art algorithms, our approach achieves better performance. Furthermore, with the human-like decision-making process, our approach incurs less than half of the computational cost.

Autonomous Exploration Under Uncertainty via Deep Reinforcement Learning on Graphs

Learning Hierarchical Graph-Based Policy for Goal-Reaching in Unknown Environments

Robot Navigation among External Autonomous Agents Through Deep Reinforcement Learning Using Graph Attention Network

Deep Reinforcement Learning with Dynamic Graphs for Adaptive Informative Path Planning

Explore with Dynamic Map: Graph Structured Reinforcement Learning

Multi-UAV Navigation for Partially Observable Communication Coverage by Graph Reinforcement Learning

Efficient Autonomous Exploration and Mapping in Unknown Environments

Goal-Driven Autonomous Exploration Through Deep Reinforcement Learning

Graph Relational Reinforcement Learning for Mobile Robot Navigation in Large-Scale Crowded Environments

Learning Crowd Behaviors in Navigation with Attention-based Spatial-Temporal Graphs

Generalized Single-Vehicle-Based Graph Reinforcement Learning for Decision-Making in Autonomous Driving

DRL-GAT-SA: Deep reinforcement learning for autonomous driving planning based on graph attention networks and simplex architecture

Fast Robot Hierarchical Exploration Based on Deep Reinforcement Learning

Deep Reinforcement Learning for Autonomous Ground Vehicle Exploration Without A-Priori Maps

Deep Reinforcement Learning-based Large-scale Robot Exploration

Autonomous Navigation of Unmanned Vehicle Through Deep Reinforcement Learning

Graph learning-based generation of abstractions for reinforcement learning

Autonomous Scene Exploration Using Experience Enhancement

Autonomous Navigation by Mobile Robot with Sensor Fusion Based on Deep Reinforcement Learning

An Enhanced Hierarchical Planning Framework for Multi-Robot Autonomous Exploration

Robot navigation in a crowd by integrating deep reinforcement learning and online planning