Quantity-Aware Coarse-to-Fine Correspondence for Image-to-Point Cloud Registration

Gongxin Yao,Yixin Xuan,Yiwei Chen,Yu Pan
DOI: https://doi.org/10.1109/jsen.2024.3454822
IF: 4.3
2024-01-01
IEEE Sensors Journal
Abstract:Image-to-point cloud registration aims to determine the relative camera posebetween an RGB image and a reference point cloud, serving as a general solutionfor locating 3D objects from 2D observations. Matching individual points withpixels can be inherently ambiguous due to modality gaps. To address thischallenge, we propose a framework to capture quantity-aware correspondencesbetween local point sets and pixel patches and refine the results at both thepoint and pixel levels. This framework aligns the high-level semantics of pointsets and pixel patches to improve the matching accuracy. On a coarse scale, theset-to-patch correspondence is expected to be influenced by the quantity of 3Dpoints. To achieve this, a novel supervision strategy is proposed to adaptivelyquantify the degrees of correlation as continuous values. On a finer scale,point-to-pixel correspondences are refined from a smaller search space througha well-designed scheme, which incorporates both resampling and quantity-awarepriors. Particularly, a confidence sorting strategy is proposed toproportionally select better correspondences at the final stage. Leveraging theadvantages of high-quality correspondences, the problem is successfullyresolved using an efficient Perspective-n-Point solver within the framework ofrandom sample consensus (RANSAC). Extensive experiments on the KITTI Odometryand NuScenes datasets demonstrate the superiority of our method over thestate-of-the-art methods.
What problem does this paper attempt to address?