PointRegGPT: Boosting 3D Point Cloud Registration using Generative Point-Cloud Pairs for Training

Suyi Chen,Hao Xu,Haipeng Li,Kunming Luo,Guanghui Liu,Chi-Wing Fu,Ping Tan,Shuaicheng Liu

2024-07-19

Abstract:Data plays a crucial role in training learning-based methods for 3D point cloud registration. However, the real-world dataset is expensive to build, while rendering-based synthetic data suffers from domain gaps. In this work, we present PointRegGPT, boosting 3D point cloud registration using generative point-cloud pairs for training. Given a single depth map, we first apply a random camera motion to re-project it into a target depth map. Converting them to point clouds gives a training pair. To enhance the data realism, we formulate a generative model as a depth inpainting diffusion to process the target depth map with the re-projected source depth map as the condition. Also, we design a depth correction module to alleviate artifacts caused by point penetration during the re-projection. To our knowledge, this is the first generative approach that explores realistic data generation for indoor point cloud registration. When equipped with our approach, several recent algorithms can improve their performance significantly and achieve SOTA consistently on two common benchmarks. The code and dataset will be released on <a class="link-external link-https" href="https://github.com/Chen-Suyi/PointRegGPT" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper primarily aims to address the issue of insufficient training data in 3D point cloud registration. Specifically, the research team proposed the PointRegGPT method, a new framework that leverages generative point cloud pairs to enhance 3D point cloud registration training. The paper points out that existing real-world datasets are costly to construct and difficult to obtain high-quality annotations, while datasets based on rendering can automatically generate large amounts of data but perform poorly in practical applications due to the domain gap.

The main contributions of PointRegGPT include:
1. **Proposing a new framework** that can automatically generate training data for 3D point cloud registration from depth maps captured in real scenes.
2. **Designing a diffusion model-based pipeline**, including depth generation and depth correction modules, which effectively produce generative point cloud pairs for training.
3. **Experimentally validating the method's effectiveness**, demonstrating superior performance through multiple registration benchmarks and significantly improving the performance of several existing registration algorithms.

The specific implementation process of PointRegGPT includes the following steps:
- Given a depth map, it is first reprojected into a target depth map through random camera motion.
- The target depth map is processed using a diffusion model to enhance the data's realism.
- A depth correction module is designed to correct errors caused by point penetration issues.
- Finally, the original depth map and the generated depth map are converted into source and target point clouds, forming training data pairs for the point cloud registration task.

Through the above methods, PointRegGPT can generate large-scale, high-quality real-world point cloud registration datasets without the need for manual annotations, thereby significantly improving the performance of learning-based methods in practical application scenarios.