Line-Based 2-D–3-D Registration and Camera Localization in Structured Environments

Huai Yu,Weikun Zhen,Wen Yang,Sebastian Scherer
DOI: https://doi.org/10.1109/TIM.2020.2999137
IF: 5.6
2020-01-01
IEEE Transactions on Instrumentation and Measurement
Abstract:Accurate registration of 2-D imagery with point clouds is a key technology for image-Light Detection and Ranging (LiDAR) point cloud fusion, camera to laser scanner calibration, and camera localization. Despite continuous improvements, automatic registration of 2-D and 3-D data without using additional textured information still faces great challenges. In this article, we propose a new 2-D–3-D registration method to estimate 2-D–3-D line feature correspondences and the camera pose in untextured point clouds of structured environments. Specifically, we first use geometric constraints between vanishing points and 3-D parallel lines to compute all feasible camera rotations. Then, we utilize a hypothesis testing strategy to estimate the 2-D–3-D line correspondences and the translation vector. By checking the consistency with computed correspondences, the best rotation matrix can be found. Finally, the camera pose is further refined using nonlinear optimization with all the 2-D–3-D line correspondences. The experimental results demonstrate the effectiveness of the proposed method on the synthetic and real data set (outdoors and indoors) with repeated structures and rapid depth changes.
What problem does this paper attempt to address?