Hybrid3D: learning 3D hybrid features with point clouds and multi-view images for point cloud registration
Bangbang Yang,Zhaoyang Huang,Yijin Li,Han Zhou,Hongsheng Li,Guofeng Zhang,Hujun Bao
DOI: https://doi.org/10.1007/s11432-022-3604-6
2023-07-07
Science China Information Sciences
Abstract:In recent years, point cloud registration has achieved great success by learning geometric features with deep learning techniques. However, existing approaches that rely on pure geometric context still suffer from sensor noise and geometric ambiguities (e.g., flat or symmetric structure), which limit their robustness to real-world scenes. When 3D point clouds are constructed by RGB-D cameras, we can enhance the learned features with complementary texture information from RGB images. To this end, we propose to learn a 3D hybrid feature that fully exploits the multi-view colored images and point clouds from indoor RGB-D scene scans. Specifically, to address the discrepancy of 2D–3D observations, we design to extract informative 2D features from image planes and take only these features for fusion. Then, we utilize a novel soft-fusion module to associate and fuse hybrid features in a unified space while alleviating the ambiguities of 2D–3D feature binding. Finally, we develop a self-supervised feature scoring module customized for our multi-modal hybrid features, which significantly improves the keypoint selection quality in noisy indoor scene scans. Our method shows competitive registration performance with previous methods on two real-world datasets.
computer science, information systems,engineering, electrical & electronic