Abstract:Recently, benefiting from the rapid development of deep learning, many research on point cloud upsampling task has been successfully proposed with remarkable performance. The quality of the upsampled point clouds is also vital for the quality of the reconstructed meshes in 3D surface reconstruction task. However, most existing methods either independently train a specific upsampling network for each scale factor or heavily rely on the paired training data as the supervision information. To address these limitations which are both inefficient and impractical for storage and computation in real applications, we present a Point-Voxel Network based on Siamese Self-supervised learning for arbitrary-Scale point cloud Upsampling (S3U-PVNet), which mainly includes a Down–Up pipeline and an Up–Down pipeline, to support self-supervised point cloud upsampling with arbitrary scale factors by a single network. The core of our network is a series of stacked point-voxel feature fusion (PVFF) modules, which can effectively and efficiently extract and fuse multi-granularity features from input point clouds. Each module includes two branches, a point-based branch that represents the sparse inputs in points to adaptively learn the spatial relationship among points, and a voxel-based branch that extract features in voxels to reduce the problem of inaccurate feature extraction caused by the irregularity and sparsity of point clouds. Furthermore, we also propose an end-to-end training objective for our siamese self-supervised network encapsulating reconstruction loss and similarity loss, which considers both global shape constraint and local geometric constraint. We achieve new state-of-the-art unsupervised upsampling results on various synthetic datasets and demonstrate the generalization of the proposed method on challenging real-world data.

Upu-Snet: Siamese Network for Unsupervised Point Cloud Upsampling Based on Spatial-Aware Transformers

S3U-PVNet: Arbitrary-scale Point Cloud Upsampling Via Point-Voxel Network Based on Siamese Self-Supervised Learning

SPU-Net: Self-Supervised Point Cloud Upsampling by Coarse-to-Fine Reconstruction with Self-Projection Optimization

PU-Transformer: Point Cloud Upsampling Transformer

PU-Net: Point Cloud Upsampling Network

SSPU-Net: A Structure Sensitive Point Cloud Upsampling Network with Multi-Scale Spatial Refinement

PU-CTG: A Point Cloud Upsampling Network Using Transformer Fusion and GRU Correction

Point Cloud Upsampling via Cascaded Refinement Network

APUNet: Attention-guided Upsampling Network for Sparse and Non-Uniform Point Cloud

SSPU-Net: Self-Supervised Point Cloud Upsampling via Differentiable Rendering

A Two-Stage Transformer-Based Point Cloud Upsampling

Point cloud upsampling via a coarse-to-fine network with transformer-encoder

HFCI-PU: High-Frequency Compensation Interpolation Network for Point Cloud Upsampling

BIMS-PU: Bi-Directional and Multi-Scale Point Cloud Upsampling

Parametric Surface Constrained Upsampler Network for Point Cloud

Point Cloud Upsampling Via Disentangled Refinement

Semantic Point Cloud Upsampling

Upsampling Autoencoder for Self-Supervised Point Cloud Learning

PU-GAT: Point Cloud Upsampling with Graph Attention Network

iPUNet: Iterative Cross Field Guided Point Cloud Upsampling

Joint Point Cloud Upsampling and Cleaning with Octree-based CNNs