Abstract:Extensive studies have shown that many animals’ capability of forming spatial representations for self-localization, path planning, and navigation relies on the functionalities of place and head-direction (HD) cells in the hippocampus. Although there are numerous hippocampal modeling approaches, only a few span the wide functionalities ranging from processing raw sensory signals to planning and action generation. This paper presents a vision-based navigation system that involves generating place and HD cells through learning from visual images, building topological maps based on learned cell representations and performing navigation using hierarchical reinforcement learning. First, place and HD cells are trained from sequences of visual stimuli in an unsupervised learning fashion. A modified Slow Feature Analysis (SFA) algorithm is proposed to learn different cell types in an intentional way by restricting their learning to separate phases of the spatial exploration. Then, to extract the encoded metric information from these unsupervised learning representations, a self-organized learning algorithm is adopted to learn over the emerged cell activities and to generate topological maps that reveal the topology of the environment and information about a robot’s head direction, respectively. This enables the robot to perform self-localization and orientation detection based on the generated maps. Finally, goal-directed navigation is performed using reinforcement learning in continuous state spaces which are represented by the population activities of place cells. In particular, considering that the topological map provides a natural hierarchical representation of the environment, hierarchical reinforcement learning (HRL) is used to exploit this hierarchy to accelerate learning. The HRL works on different spatial scales, where a high-level policy learns to select subgoals and a low-level policy learns over primitive actions to specialize on the selected subgoals. Experimental results demonstrate that our system is able to navigate a robot to the desired position effectively, and the HRL shows a much better learning performance than the standard RL in solving our navigation tasks.

RDDRL: a recurrent deduction deep reinforcement learning model for multimodal vision-robot navigation

Predictive Hierarchical Reinforcement Learning for Path-Efficient Mapless Navigation with Moving Target.

Mapless Collaborative Navigation for a Multi-Robot System Based on the Deep Reinforcement Learning

Autonomous Navigation by Mobile Robot with Sensor Fusion Based on Deep Reinforcement Learning

Deep Reinforcement Learning Based Mobile Robot Navigation Using Sensor Fusion

Multigoal Visual Navigation With Collision Avoidance via Deep Reinforcement Learning

Intelligent mobile robot navigation in unknown and complex environment using reinforcement learning technique

Deep Reinforcement Learning of Map-Based Obstacle Avoidance for Mobile Robot Navigation

Robot Navigation with Interaction-based Deep Reinforcement Learning

Learning Complicated Navigation Skills from Limited Experience Via Augmenting Offline Datasets

Learning to Navigate in Indoor Environments: from Memorizing to Reasoning

Reinforcement Learned Distributed Multi-Robot Navigation With Reciprocal Velocity Obstacle Shaped Rewards

Robot Navigation with Map-Based Deep Reinforcement Learning

A Deep Safe Reinforcement Learning Approach for Mapless Navigation.

Deep reinforcement learning based mapless navigation for industrial AMRs: advancements in generalization via potential risk state augmentation

Deep reinforcement learning navigation via decision transformer in autonomous driving

Dimension-variable Mapless Navigation with Deep Reinforcement Learning

Deep Reinforcement Learning for Robot Collision Avoidance With Self-State-Attention and Sensor Fusion

Reinforcement Learning-Based Visual Navigation With Information-Theoretic Regularization

Vision-Based Robot Navigation through Combining Unsupervised Learning and Hierarchical Reinforcement Learning

Crowd-Aware Mobile Robot Navigation Based on Improved Decentralized Structured RNN via Deep Reinforcement Learning