Abstract:Extensive studies have shown that many animals’ capability of forming spatial representations for self-localization, path planning, and navigation relies on the functionalities of place and head-direction (HD) cells in the hippocampus. Although there are numerous hippocampal modeling approaches, only a few span the wide functionalities ranging from processing raw sensory signals to planning and action generation. This paper presents a vision-based navigation system that involves generating place and HD cells through learning from visual images, building topological maps based on learned cell representations and performing navigation using hierarchical reinforcement learning. First, place and HD cells are trained from sequences of visual stimuli in an unsupervised learning fashion. A modified Slow Feature Analysis (SFA) algorithm is proposed to learn different cell types in an intentional way by restricting their learning to separate phases of the spatial exploration. Then, to extract the encoded metric information from these unsupervised learning representations, a self-organized learning algorithm is adopted to learn over the emerged cell activities and to generate topological maps that reveal the topology of the environment and information about a robot’s head direction, respectively. This enables the robot to perform self-localization and orientation detection based on the generated maps. Finally, goal-directed navigation is performed using reinforcement learning in continuous state spaces which are represented by the population activities of place cells. In particular, considering that the topological map provides a natural hierarchical representation of the environment, hierarchical reinforcement learning (HRL) is used to exploit this hierarchy to accelerate learning. The HRL works on different spatial scales, where a high-level policy learns to select subgoals and a low-level policy learns over primitive actions to specialize on the selected subgoals. Experimental results demonstrate that our system is able to navigate a robot to the desired position effectively, and the HRL shows a much better learning performance than the standard RL in solving our navigation tasks.

Long-range Navigation for Autonomous Robot based on Topo-Mapping and Q-Learning

RGBD-based Image Goal Navigation with Pose Drift: A Topo-metric Graph Based Approach

Mapless Collaborative Navigation for a Multi-Robot System Based on the Deep Reinforcement Learning

NavTopo: Leveraging Topological Maps For Autonomous Navigation Of a Mobile Robot

TopoNav: Topological Navigation for Efficient Exploration in Sparse Reward Environments

Deep reinforcement learning-aided autonomous navigation with landmark generators

Topological local-metric framework for mobile robots navigation: a long term perspective

TOP-Nav: Legged Navigation Integrating Terrain, Obstacle and Proprioception Estimation

A Mapless Local Path Planning Approach Using Deep Reinforcement Learning Framework

Learning Autonomous Navigation in Unmapped and Unknown Environments

Efficient Reinforcement Learning for 3D LiDAR Navigation of Mobile Robot

Mobile Robot Navigation Based on Noisy N-Step Dueling Double Deep Q-Network and Prioritized Experience Replay

Navigation System with SLAM-based Trajectory Topological Map and Reinforcement Learning-Based Local Planner

Self-Learning Navigation Strategy of Mobile Robot

A Novel Autonomous Exploration Algorithm Via LiDAR/IMU SLAM and Hierarchical Subsystem for Mobile Robot in Unknown Indoor Environments

Mapless Navigation Based on 2D LIDAR in Complex Unknown Environments

Long-term Navigation for Autonomous Robots Based on Spatio-Temporal Map Prediction

Research on Autonomous Robots Navigation based on Reinforcement Learning

Vision-Based Robot Navigation through Combining Unsupervised Learning and Hierarchical Reinforcement Learning

Towards navigation without precise localization: Weakly supervised learning of goal-directed navigation cost map

Learning to Navigate in a VUCA Environment: Hierarchical Multi-expert Approach