PC$^2$-PU: Patch Correlation and Point Correlation for Effective Point Cloud Upsampling

Chen Long,Wenxiao Zhang,Ruihui Li,Hao Wang,Zhen Dong,Bisheng Yang

DOI: https://doi.org/10.48550/arXiv.2109.09337

2022-07-12

Abstract:Point cloud upsampling is to densify a sparse point set acquired from 3D sensors, providing a denser representation for the underlying surface. Existing methods divide the input points into small patches and upsample each patch separately, however, ignoring the global spatial consistency between patches. In this paper, we present a novel method PC$^2$-PU, which explores patch-to-patch and point-to-point correlations for more effective and robust point cloud upsampling. Specifically, our network has two appealing designs: (i) We take adjacent patches as supplementary inputs to compensate the loss structure information within a single patch and introduce a Patch Correlation Module to capture the difference and similarity between patches. (ii) After augmenting each patch's geometry, we further introduce a Point Correlation Module to reveal the relationship of points inside each patch to maintain the local spatial consistency. Extensive experiments on both synthetic and real scanned datasets demonstrate that our method surpasses previous upsampling methods, particularly with the noisy inputs. The code and data are at \url{<a class="link-external link-https" href="https://github.com/chenlongwhu/PC2-PU.git" rel="external noopener nofollow">this https URL</a>}.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper attempts to solve two main problems in point cloud upsampling:

1. **Global Spatial Consistency**: Existing point cloud upsampling methods usually divide the input points into multiple independent patches and then upsample each patch separately, but ignore the overall spatial consistency of the target surface. This leads to the generation of additional pseudo - points at the patch boundaries.

2. **Local Spatial Consistency**: Existing methods mainly focus on the conversion from sparse to dense, but do not fully consider local spatial consistency when generating dense points, resulting in the loss of fine - grained structures. In addition, these methods are very sensitive to input quality and perform poorly especially when dealing with noisy data.

To address these problems, the paper proposes a new method - PC2 - PU (Patch Correlation and Point Correlation for Effective Point Cloud Upsampling), which explores the relationships between patches and between points to achieve more effective and robust point cloud upsampling. Specifically, this method includes the following two key modules:

- **Patch Correlation Module (PaCM)**: By encoding the similarities and differences between adjacent patches, it compensates for the missing structural information in a single patch, thereby ensuring global spatial consistency.

- **Point Correlation Module (PoCM)**: By encoding the relative positional relationships between points within each patch, it maintains local spatial consistency and restores fine - grained structures.

These two modules are jointly optimized and complementary to each other, suppressing the generation of pseudo - points and eliminating noisy points, thereby achieving more efficient and robust upsampling performance. Experimental results show that this method outperforms existing methods on both synthetic datasets and real - scanned datasets, especially when dealing with noisy inputs.