Abstract:This article tackles the visual localization of unmanned aerial vehicles (UAVs) in the presence of multisource and cross-view images are involved. We present a lightweight end-to-end scene graph encoding and matching network that finds the best matches for the airborne camera views from the reference image maps. The scene graph addresses the challenges of encoding the semantic scene by aggregating the image convolutional features into global and structured semiglobal descriptors. The principal contributions of this article are as follows: First, we develop a new network architecture that embeds a nonlocal block and a modified vector of locally aggregated descriptors network (NetVLAD) into a backbone convolutional neural network. The main component of the modified NetVLAD is a cluster similarity masking graph (CSMG) encoder, which is proposed to replace the feature-cluster residuals computing in NetVLAD with cluster consensus feature aggregation and structure-aware scene graph extraction. In addition, a global descriptor is extracted by a nonlocal block to label each image with a discriminative global feature descriptor. Second, we develop a new triplet loss for the network training procedure to learn the features at different semantic levels. The proposed global descriptor and CSMG encoder are trained together according to a weighted sum of cosine triplet losses. Third, the global descriptor from the nonlocal block and semiglobal descriptor from the CSMG encoder work hierarchically for coarse-to-fine image retrieval and can achieve real-time efficiency and favorable accuracy of image searching and matching from the reference image map. We train and test the model on two challenging benchmark datasets. We also test the pretrained model on a dataset collected by a fixed-wing UAV to further evaluate the model's generalizability. The benchmark evaluations and ablation experiments show that the developed method outperforms state-of-the-art methods and achieves superior performance in the real-time matching of UAV images and reference image maps for UAV visual localization. Open-source code is available on GitHub.

Multi-Matching-Based Vision Navigation Referencing Map Tile

A new stereo matching algorithm for UAV autonomous navigation

Cost-effective Swarm Navigation System Via Close Cooperation

A Vision-based UAV Tracker Aiming at Aerial Targets

Scene Matching Navigation for UAV Based on Spatial Relationship Geometric Constraints

Visual Localization and Environment Mapping for Micro Aerial Vehicles

Multi-UAV Collaborative Absolute Vision Positioning and Navigation: A Survey and Discussion

Navigation system based on machine vision of multiple reference markers

Real-Time Cross-View Image Matching and Camera Pose Determination for Unmanned Aerial Vehicles

Unmanned Aerial Vehicle Visual Simultaneous Localization and Mapping: A Survey

A Vision-Only Relative Distance Calculation Method for Multi-UAV Systems

A survey on vision-based UAV navigation

On the Advantages of Multiple Stereo Vision Camera Designs for Autonomous Drone Navigation

Google Map Aided Visual Navigation for UAVs in GPS-denied Environment

Jointly Optimized Global-Local Visual Localization of UAVs

Differential GNSS and Vision-Based Tracking to Improve Navigation Performance in Cooperative Multi-UAV Systems

Integrated design-sense-plan architecture for autonomous geometric-semantic mapping with UAVs

A Scene Graph Encoding and Matching Network for UAV Visual Localization

Vision-model-based Real-time Localization of Unmanned Aerial Vehicle for Autonomous Structure Inspection under GPS-denied Environment

Instant Dense 3D Reconstruction-Based UAV Vision Localization

Accurate Vision-Enabled UAV Location Using Feature-Enhanced Transformer-Driven Image Matching