Abstract:This article tackles the visual localization of unmanned aerial vehicles (UAVs) in the presence of multisource and cross-view images are involved. We present a lightweight end-to-end scene graph encoding and matching network that finds the best matches for the airborne camera views from the reference image maps. The scene graph addresses the challenges of encoding the semantic scene by aggregating the image convolutional features into global and structured semiglobal descriptors. The principal contributions of this article are as follows: First, we develop a new network architecture that embeds a nonlocal block and a modified vector of locally aggregated descriptors network (NetVLAD) into a backbone convolutional neural network. The main component of the modified NetVLAD is a cluster similarity masking graph (CSMG) encoder, which is proposed to replace the feature-cluster residuals computing in NetVLAD with cluster consensus feature aggregation and structure-aware scene graph extraction. In addition, a global descriptor is extracted by a nonlocal block to label each image with a discriminative global feature descriptor. Second, we develop a new triplet loss for the network training procedure to learn the features at different semantic levels. The proposed global descriptor and CSMG encoder are trained together according to a weighted sum of cosine triplet losses. Third, the global descriptor from the nonlocal block and semiglobal descriptor from the CSMG encoder work hierarchically for coarse-to-fine image retrieval and can achieve real-time efficiency and favorable accuracy of image searching and matching from the reference image map. We train and test the model on two challenging benchmark datasets. We also test the pretrained model on a dataset collected by a fixed-wing UAV to further evaluate the model's generalizability. The benchmark evaluations and ablation experiments show that the developed method outperforms state-of-the-art methods and achieves superior performance in the real-time matching of UAV images and reference image maps for UAV visual localization. Open-source code is available on GitHub.

An Adaptively Attention-Driven Cascade Part-Based Graph Embedding Framework for UAV Object Re-Identification

Aerial-PASS: Panoramic Annular Scene Segmentation in Drone Videos

A novel dual-pooling attention module for UAV vehicle re-identification

Dynamic Screening Strategy Based on Feature Graphs for UAV Object and Group Re-Identification

Cross-View Multi-Scale Re-Identification Network in the Perspective of Ground Rotorcraft Unmanned Aerial Vehicle

Multi-resolution feature perception network for UAV person re-identification

Multi-granularity Attention in Attention for Person Re-Identification in Aerial Images

UAV-ReID: A Benchmark on Unmanned Aerial Vehicle Re-identification in Video Imagery

Learnable Cross-Scale Sparse Attention Guided Feature Fusion for UAV Object Detection

Multi-Branch Parallel Networks for Object Detection in High-Resolution UAV Remote Sensing Images

Posture Calibration Based Cross-View & Hard-Sensitive Metric Learning for UAV-Based Vehicle Re-Identification.

A Scene Graph Encoding and Matching Network for UAV Visual Localization

Self-Attention Guidance and Multiscale Feature Fusion-Based UAV Image Object Detection

Salient Part-Aligned and Keypoint Disentangling Transformer for Person Re-Identification in Aerial Imagery

UAVSeg: Dual-Encoder Cross-Scale Attention Network for UAV Images Semantic Segmentation

Hierarchical Scale Awareness for Object Detection in Unmanned Aerial Vehicle Scenes

Deep person re-identification in UAV images

Alleviating Spatial Misalignment and Motion Interference for UAV-based Video Recognition

Differentiated Attention Guided Network over Hierarchical and Aggregated Features for Intelligent UAV Surveillance

Person Re-Identification in Aerial Imagery

ECascade-RCNN: Enhanced Cascade RCNN for Multi-scale Object Detection in UAV Images