A Cross-View Geo-Localization Algorithm Using UAV Image and Satellite Image

Jiqi Fan,Enhui Zheng,Yufei He,Jianxing Yang
DOI: https://doi.org/10.3390/s24123719
IF: 3.9
2024-06-08
Sensors
Abstract:Within research on the cross-view geolocation of UAVs, differences in image sources and interference from similar scenes pose huge challenges. Inspired by multimodal machine learning, in this paper, we design a single-stream pyramid transformer network (SSPT). The backbone of the model uses the self-attention mechanism to enrich its own internal features in the early stage and uses the cross-attention mechanism in the later stage to refine and interact with different features to eliminate irrelevant interference. In addition, in the post-processing part of the model, a header module is designed for upsampling to generate heat maps, and a Gaussian weight window is designed to assign label weights to make the model converge better. Together, these methods improve the positioning accuracy of UAV images in satellite images. Finally, we also use style transfer technology to simulate various environmental changes in order to expand the experimental data, further proving the environmental adaptability and robustness of the method. The final experimental results show that our method yields significant performance improvement: The relative distance score (RDS) of the SSPT-384 model on the benchmark UL14 dataset is significantly improved from 76.25% to 84.40%, while the meter-level accuracy (MA) of 3 m, 5 m, and 20 m is increased by 12%, 12%, and 10%, respectively. For the SSPT-256 model, the RDS has been increased to 82.21%, and the meter-level accuracy (MA) of 3 m, 5 m, and 20 m has increased by 5%, 5%, and 7%, respectively. It still shows strong robustness on the extended thermal infrared (TIR), nighttime, and rainy day datasets.
engineering, electrical & electronic,instruments & instrumentation,chemistry, analytical
What problem does this paper attempt to address?
### Problems the Paper Attempts to Solve This paper primarily aims to address the cross-view geo-localization problem of Unmanned Aerial Vehicles (UAVs) in complex environments. Specifically, the paper proposes a novel Single-Stream Pyramid Transformer (SSPT) network, designed to improve the localization accuracy of UAV images within satellite images. #### Core Challenges: 1. **Image Source Discrepancy**: The differences between UAV images and satellite images pose significant challenges to localization accuracy. 2. **Similar Scene Interference**: Visually similar but geographically different scenes can interfere with localization results. #### Main Contributions: 1. **Novel SSPT Network**: Utilizes Self-Attention and Cross-Attention mechanisms to achieve feature extraction and fusion, thereby enhancing localization accuracy and efficiency. 2. **Multi-Modal Data Augmentation Strategy**: Employs advanced style transfer techniques to simulate various environmental conditions, expanding the diversity and complexity of the training dataset. 3. **Improved Head Structure**: Uses a Gaussian function as the label weight window, further optimizing the model's localization performance in complex visual environments. ### Experimental Results - On the benchmark dataset UL14, the SSPT-384 model's Relative Distance Score (RDS) improved from 76.25% to 84.40%, and the Meter-Level Accuracy (MA) increased by 12%, 12%, and 10% (3 meters, 5 meters, 20 meters) respectively. - The SSPT-256 model's RDS improved to 82.21%, and the Meter-Level Accuracy increased by 5%, 5%, and 7%. - The model also demonstrated strong robustness on extended thermal infrared (TIR), nighttime, and rainy datasets.