Abstract:Object Goal Navigation(ObjectNav) is the task that an agent need navigate to an instance of a specific category in an unseen environment through visual observations within limited time steps. This work plays a significant role in enhancing the efficiency of locating specific items in indoor spaces and assisting individuals in completing various tasks, as well as providing support for people with disabilities. To achieve efficient ObjectNav in unfamiliar environments, global perception capabilities, understanding the regularities of space and semantics in the environment layout are significant. In this work, we propose an explicit-prediction method called VLAI that utilizes visual-language alignment information to guide the agent's exploration, unlike previous navigation methods based on frontier potential prediction or egocentric map completion, which only leverage visual observations to construct semantic maps, thus failing to help the agent develop a better global perception. Specifically, when predicting long-term goals, we retrieve previously saved visual observations to obtain visual information around the frontiers based on their position on the incrementally built incomplete semantic map. Then, we apply our designed Chat Describer to this visual information to obtain detailed frontier object descriptions. The Chat Describer, a novel automatic-questioning approach deployed in Visual-to-Language, is composed of Large Language Model(LLM) and the visual-to-language model(VLM), which has visual question-answering functionality. In addition, we also obtain the semantic similarity of target object and frontier object categories. Ultimately, by combining the semantic similarity and the boundary descriptions, the agent can predict the long-term goals more accurately. Our experiments on the Gibson and HM3D datasets reveal that our VLAI approach yields significantly better results compared to earlier methods. The code is released at https://github.com/31539lab/VLAI .

CogNav: Cognitive Process Modeling for Object Goal Navigation with LLMs

ChatNav: Leveraging LLM to Zero-shot Semantic Reasoning in Object Navigation

OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Models

Co-NavGPT: Multi-Robot Cooperative Visual Semantic Navigation using Large Language Models

Cog-GA: A Large Language Models-based Generative Agent for Vision-Language Navigation in Continuous Environments

VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Model

Embodied Contrastive Learning with Geometric Consistency and Behavioral Awareness for Object Navigation

ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddings

Think Holistically, Act Down-to-Earth: A Semantic Navigation Strategy with Continuous Environmental Representation and Multi-step Forward Planning

VLAI: Exploration and exploitation based on visual-language aligned information for robotic object goal navigation

Cognitive Planning for Object Goal Navigation using Generative AI Models

Navigation with VLM framework: Go to Any Language

Cognitive Navigation by Neuro-Inspired Localization, Mapping, and Episodic Memory

Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMs

CorNav: Autonomous Agent with Self-Corrected Planning for Zero-Shot Vision-and-Language Navigation

A Navigation Cognitive System Driven by Hierarchical Spiking Neural Network.

NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning

3D-Aware Object Goal Navigation Via Simultaneous Exploration and Identification

Navi2Gaze: Leveraging Foundation Models for Navigation and Target Gazing

Visual Navigation with Multiple Goals Based on Deep Reinforcement Learning