Abstract:Object Goal Navigation(ObjectNav) is the task that an agent need navigate to an instance of a specific category in an unseen environment through visual observations within limited time steps. This work plays a significant role in enhancing the efficiency of locating specific items in indoor spaces and assisting individuals in completing various tasks, as well as providing support for people with disabilities. To achieve efficient ObjectNav in unfamiliar environments, global perception capabilities, understanding the regularities of space and semantics in the environment layout are significant. In this work, we propose an explicit-prediction method called VLAI that utilizes visual-language alignment information to guide the agent's exploration, unlike previous navigation methods based on frontier potential prediction or egocentric map completion, which only leverage visual observations to construct semantic maps, thus failing to help the agent develop a better global perception. Specifically, when predicting long-term goals, we retrieve previously saved visual observations to obtain visual information around the frontiers based on their position on the incrementally built incomplete semantic map. Then, we apply our designed Chat Describer to this visual information to obtain detailed frontier object descriptions. The Chat Describer, a novel automatic-questioning approach deployed in Visual-to-Language, is composed of Large Language Model(LLM) and the visual-to-language model(VLM), which has visual question-answering functionality. In addition, we also obtain the semantic similarity of target object and frontier object categories. Ultimately, by combining the semantic similarity and the boundary descriptions, the agent can predict the long-term goals more accurately. Our experiments on the Gibson and HM3D datasets reveal that our VLAI approach yields significantly better results compared to earlier methods. The code is released at https://github.com/31539lab/VLAI .

Embodied Contrastive Learning with Geometric Consistency and Behavioral Awareness for Object Navigation

ChatNav: Leveraging LLM to Zero-shot Semantic Reasoning in Object Navigation

Think Holistically, Act Down-to-Earth: A Semantic Navigation Strategy with Continuous Environmental Representation and Multi-step Forward Planning

3D-Aware Object Goal Navigation Via Simultaneous Exploration and Identification

CogNav: Cognitive Process Modeling for Object Goal Navigation with LLMs

BEVNav: Robot Autonomous Navigation Via Spatial-Temporal Contrastive Learning in Bird's-Eye View

Vision-and-Language Navigation via Latent Semantic Alignment Learning

Learning Navigational Visual Representations with Semantic Map Supervision

Socially Aware Object Goal Navigation with Heterogeneous Scene Representation Learning

Visual Navigation Subject to Embodied Mismatch

VLAI: Exploration and exploitation based on visual-language aligned information for robotic object goal navigation

Object Goal Navigation using Goal-Oriented Semantic Exploration

Multi-Agent Embodied Visual Semantic Navigation with Scene Prior Knowledge

Object Goal Visual Navigation Using Semantic Spatial Relationships.

Advancing Object Goal Navigation Through LLM-enhanced Object Affinities Transfer

Seeing is Believing? Enhancing Vision-Language Navigation using Visual Perturbations

ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigation

Monocular Camera-Based Point-Goal Navigation by Learning Depth Channel and Cross-Modality Pyramid Fusion

Active Perception for Visual-Language Navigation

Symmetry-aware Neural Architecture for Embodied Visual Navigation