Neural Points: Point Cloud Representation with Neural Fields for Arbitrary Upsampling

Wanquan Feng,Jin Li,Hongrui Cai,Xiaonan Luo,Juyong Zhang

DOI: https://doi.org/10.48550/arXiv.2112.04148

2022-03-31

Abstract:In this paper, we propose Neural Points, a novel point cloud representation and apply it to the arbitrary-factored upsampling task. Different from traditional point cloud representation where each point only represents a position or a local plane in the 3D space, each point in Neural Points represents a local continuous geometric shape via neural fields. Therefore, Neural Points contain more shape information and thus have a stronger representation ability. Neural Points is trained with surface containing rich geometric details, such that the trained model has enough expression ability for various shapes. Specifically, we extract deep local features on the points and construct neural fields through the local isomorphism between the 2D parametric domain and the 3D local patch. In the final, local neural fields are integrated together to form the global surface. Experimental results show that Neural Points has powerful representation ability and demonstrate excellent robustness and generalization ability. With Neural Points, we can resample point cloud with arbitrary resolutions, and it outperforms the state-of-the-art point cloud upsampling methods. Code is available at <a class="link-external link-https" href="https://github.com/WanquanF/NeuralPoints" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper attempts to solve several key problems in point cloud representation and up - sampling tasks:

1. **Limitations of traditional point cloud representation**: In traditional point cloud representation, each point only represents a position or local plane in 3D space. This representation is limited by resolution and cannot fully capture complex geometric shapes and details. Therefore, traditional methods have limitations in representational power and detail expression.

2. **Limitations of point cloud up - sampling**: Existing point cloud up - sampling methods usually adopt a "discrete - to - discrete" strategy, that is, predicting a higher - resolution point cloud from a low - resolution input point cloud. Although this method can improve the resolution of the point cloud, it is still limited by the nature of discrete representation and cannot overcome the limitations of current point cloud representation.

To solve the above problems, the paper proposes **Neural Points**, a new type of point cloud representation method with the following characteristics:

- **Continuous representation**: Different from traditional methods, each point in Neural Points represents a locally continuous geometric shape through a neural field. This enables Neural Points to contain more shape information and thus has stronger representational power.

- **Arbitrary - multiple up - sampling**: Since Neural Points is a continuous representation method, it can be naturally applied to up - sampling tasks at any multiple, not just the discrete conversion from low - resolution to high - resolution.

- **Strong generalization ability**: Neural Points can handle various inputs and shows excellent robustness and generalization ability in experiments.

Specifically, the core idea of Neural Points is to represent each point as a local neural field, and these local neural fields are constructed through a bijective mapping between the 2D parameter domain and the 3D local surface patch. Eventually, all local neural fields are integrated into a global continuous surface. This method not only improves the representational power of the point cloud but also enables resampling at any resolution, thus achieving better performance than existing methods in point cloud up - sampling tasks.

The paper verifies the effectiveness and superiority of Neural Points through experiments, especially in handling complex geometric shapes and details.