A coarse-to-fine visual geo-localization method for GNSS-denied UAV with oblique-view imagery
Qin Ye,Junqi Luo,Yi Lin
DOI: https://doi.org/10.1016/j.isprsjprs.2024.05.006
IF: 12.7
2024-05-18
ISPRS Journal of Photogrammetry and Remote Sensing
Abstract:Visual geo-localization enables the UAV to know its position in GNSS-denied application scenarios, such as disaster emergency response, forest inventory management and indoor facility monitoring. However, there are still some challenges in using oblique-view imagery, especially for UAV images with large oblique viewing angles, to perform visual geo-localization. On the one hand, using geotags of references as image-level geo-localization results is insufficient for oblique-view imagery, because the UAV is often situated at a distance and not directly over the observation area. On the other hand, the obvious differences in viewpoint, viewing angle, and scale between UAV oblique-view images and geo-tagged orthographic reference images severely limit the accuracy of all three steps in pixel-level geo-localization, including image-based retrieval, matching and space resection. In this study, we proposed a coarse-to-fine visual geo-localization method to address these challenges for UAV position determination using oblique-view imagery, assisted by orthographic references. Firstly, we proposed a novel image-level geo-localization method by employing Segments Direction Statistics (SDS) feature patterns and bilateral retrieval strategy. Secondly, we matched UAV oblique-view images with geo-tagged orthographic reference images using feature points and descriptors extracted by the improved SuperPoint network, incorporating the Discrete Gaussian Pyramid Module (DGPM) and the Two-stage Matching Results Refinement Module (TMRRM). Thirdly, based on the matched points with 2D pixel coordinates and 3D ground coordinates, we adopted the triangular pyramid constrained resection method with angle similarity constraints to accurately determine the pixel-level position of UAV. We conducted experiments using a publicly available multi-view images dataset (University-1652), as well as two self-collected datasets of oblique-view images captured by UAVs. Experiments indicate that the localization error of the proposed method is less than 0.1 m for applications with the flight height about 200 m. Meanwhile, the error is about 10 m for UAVs with extremely high flight height (more than 4 km) and large oblique viewing angle (more than 60°). And the average time cost of the proposed method is about 400 ms. Therefore, experimental results demonstrated the effectiveness and efficiency of the proposed method in automatically obtaining reliable UAV positions using oblique-view imagery, as well as its robustness for various UAV oblique-view image-based applications.
imaging science & photographic technology,remote sensing,geography, physical,geosciences, multidisciplinary