Abstract:This article tackles the visual localization of unmanned aerial vehicles (UAVs) in the presence of multisource and cross-view images are involved. We present a lightweight end-to-end scene graph encoding and matching network that finds the best matches for the airborne camera views from the reference image maps. The scene graph addresses the challenges of encoding the semantic scene by aggregating the image convolutional features into global and structured semiglobal descriptors. The principal contributions of this article are as follows: First, we develop a new network architecture that embeds a nonlocal block and a modified vector of locally aggregated descriptors network (NetVLAD) into a backbone convolutional neural network. The main component of the modified NetVLAD is a cluster similarity masking graph (CSMG) encoder, which is proposed to replace the feature-cluster residuals computing in NetVLAD with cluster consensus feature aggregation and structure-aware scene graph extraction. In addition, a global descriptor is extracted by a nonlocal block to label each image with a discriminative global feature descriptor. Second, we develop a new triplet loss for the network training procedure to learn the features at different semantic levels. The proposed global descriptor and CSMG encoder are trained together according to a weighted sum of cosine triplet losses. Third, the global descriptor from the nonlocal block and semiglobal descriptor from the CSMG encoder work hierarchically for coarse-to-fine image retrieval and can achieve real-time efficiency and favorable accuracy of image searching and matching from the reference image map. We train and test the model on two challenging benchmark datasets. We also test the pretrained model on a dataset collected by a fixed-wing UAV to further evaluate the model's generalizability. The benchmark evaluations and ablation experiments show that the developed method outperforms state-of-the-art methods and achieves superior performance in the real-time matching of UAV images and reference image maps for UAV visual localization. Open-source code is available on GitHub.

VL-MFL: UAV Visual Localization Based on Multisource Image Feature Learning

LODM: Large-scale Online Dense Mapping for UAV

Jointly Optimized Global-Local Visual Localization of UAVs

Leveraging Map Retrieval and Alignment for Robust UAV Visual Geo-Localization

AerialVL: A Dataset, Baseline and Algorithm Framework for Aerial-based Visual Localization with Reference Map

Monocular-GPS Fusion 3D Object Detection for UAVs

LSVL: Large-scale season-invariant visual localization for UAVs

Enhancing the Localization Accuracy of UAV Images under GNSS Denial Conditions

A Scene Graph Encoding and Matching Network for UAV Visual Localization

Direction-Guided Multiscale Feature Fusion Network for Geo-Localization

From Satellite to Ground: Satellite Assisted Visual Localization with Cross-view Semantic Matching

Drone Referring Localization: An Efficient Heterogeneous Spatial Feature Interaction Method For UAV Self-Localization

UAV's Status Is Worth Considering: A Fusion Representations Matching Method for Geo-Localization

UAV Geo-Localization Dataset and Method Based on Cross-View Matching

Visual Localization and Environment Mapping for Micro Aerial Vehicles

UAV’s Status Is Worth Considering: A Fusion Representations Matching Method for Geo-Localization

LLFE: A Novel Learning Local Features Extraction for UAV Navigation Based on Infrared Aerial Image and Satellite Reference Image Matching

A 2D Georeferenced Map Aided Visual-Inertial System for Precise UAV Localization

Robust Visual-Lidar Simultaneous Localization and Mapping System for UAV

Unmanned Visual Localization Based on Satellite and Image Fusion

UAV-VisLoc: A Large-scale Dataset for UAV Visual Localization