Abstract:Visual object navigation is an essential task of embodied AI, which is letting the agent navigate to the goal object under the user's demand. Previous methods often focus on single-object navigation. However, in real life, human demands are generally continuous and multiple, requiring the agent to implement multiple tasks in sequence. These demands can be addressed by repeatedly performing previous single task methods. However, by dividing multiple tasks into several independent tasks to perform, without the global optimization between different tasks, the agents' trajectories may overlap, reducing the efficiency of navigation. In this paper, we propose an efficient reinforcement learning framework with a hybrid policy for multi-object navigation, aiming to maximally eliminate noneffective actions. First, the visual observations are embedded to detect the semantic entities (such as objects). And the detected objects are memorized and projected into semantic maps, which can also be regarded as a long-term memory of the observed environment. Then a hybrid policy consisting of exploration and long-term planning strategies is proposed to predict the potential target position. In particular, when the target is directly oriented, the policy function makes long-term planning for the target based on the semantic map, which is implemented by a sequence of motion actions. In the alternative, when the target is not oriented, the policy function estimates an object's potential position toward exploring the most possible objects (positions) that have close relations to the target. The relation between different objects is obtained with prior knowledge, which is used to predict the potential target position by integrating with the memorized semantic map. And then a path to the potential target is planned by the policy function. We evaluate our proposed method on two large-scale 3D realistic environment datasets, Gibson and- Matterport3D, and the experimental results demonstrate the effectiveness and generalization of the proposed method.

Optimal Policy Replay: A Simple Method to Reduce Catastrophic Forgetting in Target Incremental Visual Navigation

Memory-based soft actor–critic with prioritized experience replay for autonomous navigation

Replay-enhanced Continual Reinforcement Learning

Leveraging Efficiency Through Hybrid Prioritized Experience Replay in Door Environment.

Learning Complicated Navigation Skills from Limited Experience Via Augmenting Offline Datasets

Spiking Reinforcement Learning with Memory Ability for Mapless Navigation

Multigoal Visual Navigation With Collision Avoidance via Deep Reinforcement Learning

Map-based Experience Replay: A Memory-Efficient Solution to Catastrophic Forgetting in Reinforcement Learning

A target-driven visual navigation method based on intrinsic motivation exploration and space topological cognition

Navigation Command Matching for Vision-based Autonomous Driving

The Empirical Impact of Forgetting and Transfer in Continual Visual Odometry

ONER: Online Experience Replay for Incremental Anomaly Detection

Incremental Learning for Autonomous Navigation of Mobile Robots based on Deep Reinforcement Learning

Deep Reinforcement Learning Visual Navigation Model Integrating Memory-prediction Mechanism

Predictive Experience Replay for Continual Visual Control and Forecasting

Memory-Based Parameterized Skills Learning for Mapless Visual Navigation.

DGMem: Learning Visual Navigation Policy Without Any Labels by Dynamic Graph Memory

Graph Attention Memory for Visual Navigation.

Cognitive Navigation for Intelligent Mobile Robots: A Learning-Based Approach with Topological Memory Configuration

Deep Visual MPC-Policy Learning for Navigation

Multi-Object Navigation Using Potential Target Position Policy Function