Abstract:By formulating the data generation as a sequence procedure of denoising autoencoding, diffusion models have achieved superior in-painting performance on image data and beyond. Nevertheless, it is not trivial when capitalizing on diffusion models to generate missing 3D points. The difficulty originates from the intrinsic structure where 3D point cloud is a set of unordered and irregular coordinates. That motivates us to delve into the 3D structural information for designing point cloud encoder-decoder and shape latent generator, to precisely formulate the latent distribution of the complete point cloud and partial observation. In this paper, we propose Point cloud completion with Latent Diffusion Models (PointLDM), a new approach that leverages the conditional denoising diffusion probabilistic modeling (DDPM) in the 3D latent space for shape reconstruction. The architecture of PointLDM consists of a transformer-based variational auto-encoder (VAE) to model the complete shape latent, and a diffusion network for shape latent prediction. The encoder of VAE exploits both of global shape latent and local point features in shape distribution learning. With the learnt shape latent, the decoder first decodes the shape latent into coarse points, and then recovers the fine-grained details around each coarse point by deforming a 2D grid. To reconstruct the shape latent from partial observation, the diffusion network treats the partial observation as the conditional input and generates the shape latent via DDPM. Extensive experiments conducted on MVP, Completion3D, and KITTI quantitatively and qualitatively demonstrate the efficacy of PointLDM over the state-of-the-art shape completion approaches.

Autoregressive 3D Shape Completion via Sphere-Guided Disentangled Representation.

ShapeFormer: Transformer-based Shape Completion Via Sparse Representation

Shape Completion with Azimuthal Rotations Using Spherical Gidding-Based Invariant and Equivariant Network

Learning 3D Shape Latent for Point Cloud Completion

3D shape completion via deep learning: a method survey

POC-SLT: Partial Object Completion with SDF Latent Transformers

SC-Diff: 3D Shape Completion with Latent Diffusion Models

VQ-DcTr: Vector-Quantized Autoencoder with Dual-channel Transformer Points Splitting for 3D Point Cloud Completion

Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis

Shape Completion from a Single RGBD Image.

3D Shape Completion on Unseen Categories:A Weakly-supervised Approach

3D Shape Generation and Completion through Point-Voxel Diffusion

Point cloud completion network for 3D shapes with morphologically diverse structures

3D Shape Completion on Unseen Categories: A Weakly-Supervised Approach

3DPCTN: Two 3D Local-Object Point-Cloud-Completion Transformer Networks Based on Self-Attention and Multi-Resolution

SD-Net: Spatially-Disentangled Point Cloud Completion Network

Unsupervised Semantic Scene Reconstruction via Transformer-Based Quantized Vector Reconstruction and Autoregressive Completion

Self-supervised Shape Completion via Involution and Implicit Correspondences

Spherical Transformer for LiDAR-based 3D Recognition

NeuSDFusion: A Spatial-Aware Generative Model for 3D Shape Completion, Reconstruction, and Generation

Point Cloud Completion Via Skeleton-Detail Transformer