Abstract:Autonomous navigation substantially depends on memory mechanisms for enhancing path optimality. However, the dynamic nature of environments can cause inconsistencies between the stored scene memory and real-time perception, leading to potentially catastrophic navigation errors. Existing memory structures often fall short in addressing these challenges, as they typically only account for object-level dynamics and falter when faced with long-term alterations in environmental structure. To counter these constraints, this paper presents a learning-based framework designed to construct and maintain a geometric memory, thereby facilitating improved visuomotor navigation under structural dynamics. This proposed framework employs visual inputs to construct a geometric representation of the environment, and identifies structural changes by assessing the consistency between the established memory and instantaneous perception. To update the geometric memory efficiently, we introduce a memory updater grounded in a structure storage pool. Furthermore, a two-phase hierarchical planner is proposed to decompose navigation tasks and formulate smooth navigation strategies. Experimental results from photorealistic simulations underscore the efficacy of the proposed system in managing long-term dynamics and navigation control. The effectiveness of the proposed system is further corroborated through deployment and testing in real-world environments. Note to Practitioners—Classic geometry-based navigation methods hinge on the construction of a global map for spatial reasoning and optimized robot control. Within a learning-based pipeline, the dependence on memory information becomes critical for enabling robots to develop spatial and temporal awareness. While the maintenance of memory structures can significantly expand the field of the robot’s perception in time and space, discrepancies between historical memory and real-time perception in dynamic environments can lead to misguided decisions. Unlike most existing research that addresses short-term dynamic issues at the object level, this paper centers on long-term, large-scale dynamics precipitated by changes in environmental structure. We put forward a learning-based framework that incorporates dynamic perception, map maintenance, and hierarchical navigation. The experimental results highlight the efficiency and real-time processing capability of this method in handling structural dynamics, hence enhancing navigation efficiency.

Memory-based soft actor–critic with prioritized experience replay for autonomous navigation

Cognitive Navigation by Neuro-Inspired Localization, Mapping, and Episodic Memory

Spiking Reinforcement Learning with Memory Ability for Mapless Navigation

A Soft Actor-Critic Deep Reinforcement-Learning-Based Robot Navigation Method Using LiDAR

Autonomous navigation of mobile robots in unknown environments using off-policy reinforcement learning with curriculum learning

Attention-based advantage actor-critic algorithm with prioritized experience replay for complex 2-D robotic motion planning

A neuro-cognitive robot for spatial navigation

Mobile Robot Navigation Based on Noisy N-Step Dueling Double Deep Q-Network and Prioritized Experience Replay

Cognitive Memory and Mapping in a Brain-Like System for Robotic Navigation

Real-Time Navigation In Dynamic Human Environments Using Optimal Reciprocal Collision Avoidance

Navigation in Unknown Dynamic Environments Based on Deep Reinforcement Learning

Learning Autonomous Navigation in Unmapped and Unknown Environments

Deep Reinforcement Learning for Autonomous Driving with an Auxiliary Actor Discriminator

Integrated model of cerebellal supervised learning and basal ganglia's reinforcement learning for mobile robot behavioral decision-making

A Novel Neural Multi-Store Memory Network for Autonomous Visual Navigation in Unknown Environment

Deep reinforcement learning-aided autonomous navigation with landmark generators

Cognitive Navigation for Intelligent Mobile Robots: A Learning-Based Approach with Topological Memory Configuration

Online Geometric Memory Generation and Maintenance for Visuomotor Navigation in Structural Dynamic Environments

MeSA-DRL: Memory-Enhanced Deep Reinforcement Learning for Advanced Socially Aware Robot Navigation in Crowded Environments

A priority experience replay actor-critic algorithm using self-attention mechanism for strategy optimization of discrete problems

Multigoal Visual Navigation With Collision Avoidance via Deep Reinforcement Learning