Abstract:Developing agents capable of navigating to a target location based on language instructions and visual information, known as vision-language navigation (VLN), has attracted widespread interest. Most research has focused on ground-based agents, while UAV-based VLN remains relatively underexplored. Recent efforts in UAV vision-language navigation predominantly adopt ground-based VLN settings, relying on predefined discrete action spaces and neglecting the inherent disparities in agent movement dynamics and the complexity of navigation tasks between ground and aerial environments. To address these disparities and challenges, we propose solutions from three perspectives: platform, benchmark, and methodology. To enable realistic UAV trajectory simulation in VLN tasks, we propose the OpenUAV platform, which features diverse environments, realistic flight control, and extensive algorithmic support. We further construct a target-oriented VLN dataset consisting of approximately 12k trajectories on this platform, serving as the first dataset specifically designed for realistic UAV VLN tasks. To tackle the challenges posed by complex aerial environments, we propose an assistant-guided UAV object search benchmark called UAV-Need-Help, which provides varying levels of guidance information to help UAVs better accomplish realistic VLN tasks. We also propose a UAV navigation LLM that, given multi-view images, task descriptions, and assistant instructions, leverages the multimodal understanding capabilities of the MLLM to jointly process visual and textual information, and performs hierarchical trajectory generation. The evaluation results of our method significantly outperform the baseline models, while there remains a considerable gap between our results and those achieved by human operators, underscoring the challenge presented by the UAV-Need-Help task.

Deep Learning Based Positioning Scheme for Single UAV-Assisted VLC Systems

Autonomous Obstacle Avoidance and Communication Capacity Optimization for UAV-Assisted VLC Systems

Trajectory Planning for Cross-Interface UAV-Assisted VLC Systems

Small Unmanned Aerial Vehicle Visual System for Ground Moving Target Positioning

Deep Learning for Channel Tracking in IRS-Assisted UAV Communication Systems

Robust Visual-Lidar Simultaneous Localization and Mapping System for UAV

Location-aware Predictive Beamforming for UAV Communications: A Deep Learning Approach

VL-MFL: UAV Visual Localization Based on Multisource Image Feature Learning

Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodology

Jointly Optimized Global-Local Visual Localization of UAVs

Research on optimal deep learning‐based formation flight positioning of UAV

Deep-learning based autonomous-exploration for UAV navigation

Deep Reinforcement Learning for UAV Navigation Through Massive MIMO Technique

A Visible Light Positioning System Based on Particle Filter and Deep Learning

UAV Pose Estimation in GNSS-Denied Environment Assisted by Satellite Imagery Deep Learning Features

Learning-Enabled Radar-Assisted Predictive Beamforming for UAV-Aided Networks

An Underwater Positioning System for UUVs Based on LiDAR Camera and Inertial Measurement Unit

Long-Range Vision-Based UAV-assisted Localization for Unmanned Surface Vehicles

A Vision-Based Target Detection, Tracking, and Positioning Algorithm for Unmanned Aerial Vehicle

Real-Time Multi-Target Localization from Unmanned Aerial Vehicles