Learn then Match: A Fast Coarse-to-fine Depth Image-Based Indoor Localization Framework for Dark Environments Via Deep Learning and Keypoint-Based Geometry Alignment

Qing Li,Rui Cao,Jiasong Zhu,Hao Fu,Baoding Zhou,Xu Fang,Sen Jia,Shenman Zhang,Kanglin Liu,Qingquan Li

DOI: https://doi.org/10.1016/j.isprsjprs.2022.10.015

IF: 12.7

2022-01-01

ISPRS Journal of Photogrammetry and Remote Sensing

Abstract:Image-based indoor localization provides fundamental support for applications such as indoor navigation, virtual reality, and location-based services. Most research focuses on developing methods in good lighting conditions via RGB images; while for low lighting situations, especially at night, RGB-based methods cannot perform well. Depth images are promising alternative in such conditions as they record geometrical information instead of texture information, making it possible to work in low lighting scenarios. Current depth image -based methods, either retrieval-based methods or 3D registration-based methods, are inefficient due to its high computation overhead, preventing the wide applications. To address this issue, we propose a fast coarse -to-fine localization framework for dark environment via deep learning and keypoint-based geometry alignment. In the coarse localization phase, we jointly perform the depth completion and pose regression to relieve the occlusion caused appearance variance in depth images. In the refinement phase, keypoints are used instead of whole depth image points under the ICP alignment framework to increase the localization efficiency. The keypoints are detected on depth feature maps weakly supervised with pose regression. The experiments on the open available 7Scenes dataset show that the proposed method obtain positional accuracy of 0.143 m and orientational accuracy of 5.275 degrees in average and only cost 0.8s for a single depth image. The code for the proposed work is available at https://github.com/lqing900205/KeyPointDepthLocalization