P2C: Self-Supervised Point Cloud Completion from Single Partial Clouds

Ruikai Cui,Shi Qiu,Saeed Anwar,Jiawei Liu,Chaoyue Xing,Jing Zhang,Nick Barnes

DOI: https://doi.org/10.1109/iccv51070.2023.01320

2023-07-27

Abstract:Point cloud completion aims to recover the complete shape based on a partial observation. Existing methods require either complete point clouds or multiple partial observations of the same object for learning. In contrast to previous approaches, we present Partial2Complete (P2C), the first self-supervised framework that completes point cloud objects using training samples consisting of only a single incomplete point cloud per object. Specifically, our framework groups incomplete point clouds into local patches as input and predicts masked patches by learning prior information from different partial objects. We also propose Region-Aware Chamfer Distance to regularize shape mismatch without limiting completion capability, and devise the Normal Consistency Constraint to incorporate a local planarity assumption, encouraging the recovered shape surface to be continuous and complete. In this way, P2C no longer needs multiple observations or complete point clouds as ground truth. Instead, structural cues are learned from a category-specific dataset to complete partial point clouds of objects. We demonstrate the effectiveness of our approach on both synthetic ShapeNet data and real-world ScanNet data, showing that P2C produces comparable results to methods trained with complete shapes, and outperforms methods learned with multiple partial observations. Code is available at <a class="link-external link-https" href="https://github.com/CuiRuikai/Partial2Complete" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition,Graphics

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper aims to address the issue of Point Cloud Completion (PCC), specifically how to complete point cloud objects when only a single incomplete point cloud sample is available. Traditional point cloud completion methods typically require complete point clouds or multiple partial observations to train the model, which is often difficult to achieve in practical applications. This paper proposes a novel self-supervised framework—Partial2Complete (P2C), which can complete point clouds using only one incomplete point cloud sample per object during training, significantly reducing the cost of data collection and annotation.

### Main Contributions

1. **First Proposed Self-Supervised Framework**: P2C is the first self-supervised framework capable of completing point clouds using only one incomplete point cloud sample per object.
2. **Introduction of Region-Aware Chamfer Distance (RCD)**: RCD overcomes the limitations of traditional distance metrics in completion tasks by constructing local regions centered on dynamically generated skeleton points, improving completion capability.
3. **Proposed Normal Consistency Constraint (NCC)**: NCC encourages the generated points to follow the local 2D surface manifold of the incomplete point cloud by minimizing a new consistency metric, enhancing surface continuity and integrity.

### Method Overview

1. **Partial2Complete Framework**:
- Input an incomplete point cloud \( P_p \), and divide it into multiple local regions (patches).
- These patches are divided into three groups: observable region \( G_{\text{rec}} \), completion region \( G_{\text{com}} \), and latent region \( G_{\text{latent}} \).
- The encoder encodes \( G_{\text{rec}} \) to generate features \( f \), and the decoder generates the predicted point cloud \( P_c \) based on \( f \).
- The network is optimized through reconstruction loss \( L_r \) and completion loss \( L_c \) to ensure the predicted point cloud is consistent with the observable region and can complete the missing parts.
- Latent reconstruction loss \( L_f \) and normal consistency constraint \( L_{\text{ncc}} \) are introduced to further regularize the completion results.

2. **Region-Aware Chamfer Distance (RCD)**:
- By constructing local regions centered on skeleton points in the predicted and partial input point clouds, RCD can better handle the differences between observed and unobserved regions.
- RCD is defined as follows:
\[
d_{\text{RCD}}(P_p, P_c) = d_{\text{UCD}}(R_p, R_c) + d_{\text{UCD}}(R_c, R_p)
\]
where \( R_p \) and \( R_c \) represent the local regions in the partial and predicted point clouds, respectively.

3. **Normal Consistency Constraint (NCC)**:
- By estimating the normal direction of each point and calculating the normal similarity between neighboring points, NCC encourages the generated points to follow the local surface manifold.
- NCC is defined as follows:
\[
\text{NCC}(P) = \frac{1}{n} \sum_{i=1}^n \text{nc}(p_i)
\]
where \( \text{nc}(p_i) \) represents the normal similarity variance of point \( p_i \).

### Experimental Results

1. **Synthetic Dataset Evaluation**:
- On the 3D-EPN and PCN datasets, P2C outperforms existing unpaired and weakly supervised methods.
- Compared to the best unpaired method, P2C improves the CD-ℓ2 score by 2.7 on the 3D-EPN dataset.
- Compared to the best weakly supervised method, P2C improves the CD score on the 3D-EPN dataset.