PU-EdgeFormer: Edge Transformer for Dense Prediction in Point Cloud Upsampling

Dohoon Kim,Minwoo Shin,Joonki Paik

2023-05-02

Abstract:Despite the recent development of deep learning-based point cloud upsampling, most MLP-based point cloud upsampling methods have limitations in that it is difficult to train the local and global structure of the point cloud at the same time. To solve this problem, we present a combined graph convolution and transformer for point cloud upsampling, denoted by PU-EdgeFormer. The proposed method constructs EdgeFormer unit that consists of graph convolution and multi-head self-attention modules. We employ graph convolution using EdgeConv, which learns the local geometry and global structure of point cloud better than existing point-to-feature method. Through in-depth experiments, we confirmed that the proposed method has better point cloud upsampling performance than the existing state-of-the-art method in both subjective and objective aspects. The code is available at <a class="link-external link-https" href="https://github.com/dohoon2045/PU-EdgeFormer" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in the process of point cloud up - sampling, existing MLP - based methods have difficulty in training the local and global structures of point clouds simultaneously. Specifically, the paper points out that although deep - learning - based point cloud up - sampling techniques have made significant progress, most MLP - based methods have limitations in dealing with the local geometry and global structure of point clouds. To solve this problem, the authors propose a new method that combines graph convolution and Transformer, called PU - EdgeFormer.

### Specific Problem Description

1. **Balance between Local and Global Structures**:
- Existing MLP - based methods perform poorly when dealing with the local geometry and global structure of point clouds and have difficulty in capturing the features of these two aspects simultaneously.
- This results in deficiencies in the details and overall structure of the up - sampled point clouds.

2. **Sparsity and Noise of Point Clouds**:
- The acquired point clouds are usually sparse and noisy, which reduces the performance when they are directly applied to 3D tasks.
- Therefore, point cloud up - sampling in the pre - processing step becomes particularly important in order to generate dense, uniform and noise - free point clouds.

### Solution

To overcome the above problems, the authors propose PU - EdgeFormer, and its main contributions include:

1. **EdgeFormer Unit**:
- It combines graph convolution (using EdgeConv) and multi - head self - attention mechanism (Transformer).
- It learns the local geometric structure of point clouds through EdgeConv and captures the global structure of point clouds through the multi - head self - attention mechanism.

2. **Network Architecture**:
- **Encoder**: Converts low - resolution point clouds into point features.
- **Feature Expansion**: Expands point features through reshaping and MLP operations.
- **Coordinate Reconstruction**: Combines the expanded features with the original point clouds to generate high - resolution point clouds.

3. **Experimental Verification**:
- Through experiments on the PU1K dataset, it is verified that PU - EdgeFormer outperforms the existing state - of - the - art methods in the point cloud up - sampling task.
- The experimental results show that PU - EdgeFormer performs well in both subjective and objective evaluation metrics, especially in Chamfer distance and Hausdorff distance.

### Summary

This paper successfully solves the problem of balancing local and global structures in point cloud up - sampling by introducing the EdgeFormer unit and demonstrates its superior performance under various experimental settings. Future work will further study how to improve the edge smoothing problem to further enhance the effect of point cloud up - sampling.