Abstract:Extensive studies have shown that many animals’ capability of forming spatial representations for self-localization, path planning, and navigation relies on the functionalities of place and head-direction (HD) cells in the hippocampus. Although there are numerous hippocampal modeling approaches, only a few span the wide functionalities ranging from processing raw sensory signals to planning and action generation. This paper presents a vision-based navigation system that involves generating place and HD cells through learning from visual images, building topological maps based on learned cell representations and performing navigation using hierarchical reinforcement learning. First, place and HD cells are trained from sequences of visual stimuli in an unsupervised learning fashion. A modified Slow Feature Analysis (SFA) algorithm is proposed to learn different cell types in an intentional way by restricting their learning to separate phases of the spatial exploration. Then, to extract the encoded metric information from these unsupervised learning representations, a self-organized learning algorithm is adopted to learn over the emerged cell activities and to generate topological maps that reveal the topology of the environment and information about a robot’s head direction, respectively. This enables the robot to perform self-localization and orientation detection based on the generated maps. Finally, goal-directed navigation is performed using reinforcement learning in continuous state spaces which are represented by the population activities of place cells. In particular, considering that the topological map provides a natural hierarchical representation of the environment, hierarchical reinforcement learning (HRL) is used to exploit this hierarchy to accelerate learning. The HRL works on different spatial scales, where a high-level policy learns to select subgoals and a low-level policy learns over primitive actions to specialize on the selected subgoals. Experimental results demonstrate that our system is able to navigate a robot to the desired position effectively, and the HRL shows a much better learning performance than the standard RL in solving our navigation tasks.

Inverse reinforcement learning-based time-dependent A* planner for human-aware robot navigation with local vision

Real-Time Navigation In Dynamic Human Environments Using Optimal Reciprocal Collision Avoidance

Robot Path Planning for Human Search in Indoor Environments

Deep Reinforcement Learning for Localizability-Enhanced Navigation in Dynamic Human Environments

Visual Local Path Planning Based on Deep Reinforcement Learning

Learning to Roam Free from Small-Space Autonomous Driving with A Path Planner

Robot path planner based on deep reinforcement learning and the seeker optimization algorithm

Social Navigation Planning Based on People's Awareness of Robots

Deep Local Trajectory Replanning and Control for Robot Navigation

Deep reinforcement learning-aided autonomous navigation with landmark generators

Socially compliant mobile robot navigation via inverse reinforcement learning

Socially Aware Motion Planning with Deep Reinforcement Learning

Adaptive Social Force Window Planner with Reinforcement Learning

Efficient and Trustworthy Social Navigation Via Explicit and Implicit Robot-Human Communication

Robot Navigation in Unknown and Cluttered Workspace with Dynamical System Modulation in Starshaped Roadmap

Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Driving

An Efficient Planning Method for Autonomous Navigation of a Wheeled-Robot based on Deep Reinforcement Learning

Traversability-Aware Legged Navigation by Learning from Real-World Visual Data

Vision-Based Robot Navigation through Combining Unsupervised Learning and Hierarchical Reinforcement Learning

Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation

HDPlanner: Advancing Autonomous Deployments in Unknown Environments through Hierarchical Decision Networks