F3-Net: Multiview Scene Matching for Drone-Based Geo-Localization

Bo Sun,Ganchao Liu,Yuan

DOI: https://doi.org/10.1109/tgrs.2023.3278257

IF: 8.2

2023-01-01

IEEE Transactions on Geoscience and Remote Sensing

Abstract:Scene matching involves establishing a mapping relationship between heterogeneous images, which is crucial for drone visual geo-localization. However, it poses a significant challenge for multiview images such as those captured by drones and satellites. To address this issue, this article proposes an end-to-end geo-localization framework named F3-Net for calculating the similarity of multisource and multiview images. The key contributions of F3-Net are as follows: 1) the split and fusion (SF) module is designed to fully exploit the features through the global self-attention mechanism; 2) to improve the multiview semantic features, a target feature enhancement (TFE) module is introduced, based on the principle of invariance target semantic consistency; and 3) after multiview feature learning, a feature alignment and unity (FAU) module with Earthmover (EM) distance is used to calculate the similarity of nonaligned features. F3-Net fully exploits the multisource image feature correspondence and multiview image semantic consistency. Different from the traditional Siamese network, the features of multiview images are regarded as a probability distribution, so F3-Net can quantify and eliminate the feature differences of multiview images in the learning process. Experiments show that F3-Net can effectively overcome multiview changes and achieve high accuracy on the University-1652 dataset.