Abstract:Extensive studies have shown that many animals’ capability of forming spatial representations for self-localization, path planning, and navigation relies on the functionalities of place and head-direction (HD) cells in the hippocampus. Although there are numerous hippocampal modeling approaches, only a few span the wide functionalities ranging from processing raw sensory signals to planning and action generation. This paper presents a vision-based navigation system that involves generating place and HD cells through learning from visual images, building topological maps based on learned cell representations and performing navigation using hierarchical reinforcement learning. First, place and HD cells are trained from sequences of visual stimuli in an unsupervised learning fashion. A modified Slow Feature Analysis (SFA) algorithm is proposed to learn different cell types in an intentional way by restricting their learning to separate phases of the spatial exploration. Then, to extract the encoded metric information from these unsupervised learning representations, a self-organized learning algorithm is adopted to learn over the emerged cell activities and to generate topological maps that reveal the topology of the environment and information about a robot’s head direction, respectively. This enables the robot to perform self-localization and orientation detection based on the generated maps. Finally, goal-directed navigation is performed using reinforcement learning in continuous state spaces which are represented by the population activities of place cells. In particular, considering that the topological map provides a natural hierarchical representation of the environment, hierarchical reinforcement learning (HRL) is used to exploit this hierarchy to accelerate learning. The HRL works on different spatial scales, where a high-level policy learns to select subgoals and a low-level policy learns over primitive actions to specialize on the selected subgoals. Experimental results demonstrate that our system is able to navigate a robot to the desired position effectively, and the HRL shows a much better learning performance than the standard RL in solving our navigation tasks.

Monocular Robot Navigation with Self-Supervised Pretrained Vision Transformers

VANP: Learning Where to See for Navigation with Self-Supervised Vision-Action Pre-Training

MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer

Robot Perception enables Complex Navigation Behavior via Self-Supervised Learning

Self-supervised Pretraining and Finetuning for Monocular Depth and Visual Odometry

Online Robot Navigation and Manipulation with Distilled Vision-Language Models

Distilling Self-Supervised Vision Transformers for Weakly-Supervised Few-Shot Classification & Segmentation

Transformer-Based Self-Supervised Monocular Depth and Visual Odometry

Depth-Aware Vision-and-Language Navigation Using Scene Query Attention Network

Vision-Based Robot Navigation through Combining Unsupervised Learning and Hierarchical Reinforcement Learning

Autonomous Navigation for Mobile Robots with Weakly-Supervised Segmentation Network

Vision Transformers for End-to-End Vision-Based Quadrotor Obstacle Avoidance

Real-World Robot Learning with Masked Visual Pre-training

Self-supervised Vision Transformers for Land-cover Segmentation and Classification

Transformers in Unsupervised Structure-from-Motion

See What the Robot Can't See: Learning Cooperative Perception for Visual Navigation

Rethinking Vision Transformer Through Human–object Interaction Detection

An Empirical Study of Training Self-Supervised Vision Transformers

How to Train Vision Transformer on Small-scale Datasets?

The Optimization of Localization and Navigation for Vision-based Robot

A Simple Single-Scale Vision Transformer for Object Localization and Instance Segmentation