Abstract:Object Goal Navigation(ObjectNav) is the task that an agent need navigate to an instance of a specific category in an unseen environment through visual observations within limited time steps. This work plays a significant role in enhancing the efficiency of locating specific items in indoor spaces and assisting individuals in completing various tasks, as well as providing support for people with disabilities. To achieve efficient ObjectNav in unfamiliar environments, global perception capabilities, understanding the regularities of space and semantics in the environment layout are significant. In this work, we propose an explicit-prediction method called VLAI that utilizes visual-language alignment information to guide the agent's exploration, unlike previous navigation methods based on frontier potential prediction or egocentric map completion, which only leverage visual observations to construct semantic maps, thus failing to help the agent develop a better global perception. Specifically, when predicting long-term goals, we retrieve previously saved visual observations to obtain visual information around the frontiers based on their position on the incrementally built incomplete semantic map. Then, we apply our designed Chat Describer to this visual information to obtain detailed frontier object descriptions. The Chat Describer, a novel automatic-questioning approach deployed in Visual-to-Language, is composed of Large Language Model(LLM) and the visual-to-language model(VLM), which has visual question-answering functionality. In addition, we also obtain the semantic similarity of target object and frontier object categories. Ultimately, by combining the semantic similarity and the boundary descriptions, the agent can predict the long-term goals more accurately. Our experiments on the Gibson and HM3D datasets reveal that our VLAI approach yields significantly better results compared to earlier methods. The code is released at https://github.com/31539lab/VLAI .

RILA: Reflective and Imaginative Language Agent for Zero-Shot Semantic Audio-Visual Navigation

ChatNav: Leveraging LLM to Zero-shot Semantic Reasoning in Object Navigation

Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigation

VLAI: Exploration and exploitation based on visual-language aligned information for robotic object goal navigation

Vision-Language Navigation Policy Learning and Adaptation

NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation

Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMs

TINA: Think, Interaction, and Action Framework for Zero-Shot Vision Language Navigation

Aerial Vision-and-Language Navigation via Semantic-Topo-Metric Representation Guided LLM Reasoning

Perceive, Reflect, and Plan: Designing LLM Agent for Goal-Directed City Navigation without Instructions

Cog-GA: A Large Language Models-based Generative Agent for Vision-Language Navigation in Continuous Environments

Vision-and-Language Navigation via Latent Semantic Alignment Learning

Active Visual Information Gathering for Vision-Language Navigation

L3MVN: Leveraging Large Language Models for Visual Target Navigation

VLN-Game: Vision-Language Equilibrium Search for Zero-Shot Semantic Navigation

Navigation with VLM framework: Go to Any Language

Actional Atomic-Concept Learning for Demystifying Vision-Language Navigation

NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Models

VELMA: Verbalization Embodiment of LLM Agents for Vision and Language Navigation in Street View

Reinforced Structured State-Evolution for Vision-Language Navigation

ImagineNav: Prompting Vision-Language Models as Embodied Navigator through Scene Imagination