2D-3DMatchingNet: Multimodal Point Completion with 2D Geometry Matching

Zaiyang Yu,Liping Zhang,Shuang Li,Yugui Zhang,Xin Ning
DOI: https://doi.org/10.1109/hpbdis53214.2021.9658452
2021-12-05
Abstract:As the performance of RGB-D cameras improves, varieties of real-world partial scans are used in target recognition, face matching, scene segmentation, etc. Meanwhile, sparse and locally missing real-world scans greatly affect the performance of the model. Point cloud completion replaces such sparse and missing scans with dense and uniform point clouds. However, most of the work focuses only on the complete effect of the model in 3D dimension but neglects the completion exterior appearance in 2D space, which we define as 2D3D-mismatch problem. To solve this problem, we not only extract the location features in 3D dimension but also calibrate the point cloud shape features in 2D space. Meanwhile, the proposed method is more robust to real-world point cloud scans and provides multimodal completion results by leveraging an unsupervised strategy: Generative Adversarial Network (GAN). Extensive experiments demonstrate that our method outperforms other unsupervised point cloud completion methods. Notably, our approach generates smoother surface and robustness on out-domain data.
What problem does this paper attempt to address?