Abstract:By formulating the data generation as a sequence procedure of denoising autoencoding, diffusion models have achieved superior in-painting performance on image data and beyond. Nevertheless, it is not trivial when capitalizing on diffusion models to generate missing 3D points. The difficulty originates from the intrinsic structure where 3D point cloud is a set of unordered and irregular coordinates. That motivates us to delve into the 3D structural information for designing point cloud encoder-decoder and shape latent generator, to precisely formulate the latent distribution of the complete point cloud and partial observation. In this paper, we propose Point cloud completion with Latent Diffusion Models (PointLDM), a new approach that leverages the conditional denoising diffusion probabilistic modeling (DDPM) in the 3D latent space for shape reconstruction. The architecture of PointLDM consists of a transformer-based variational auto-encoder (VAE) to model the complete shape latent, and a diffusion network for shape latent prediction. The encoder of VAE exploits both of global shape latent and local point features in shape distribution learning. With the learnt shape latent, the decoder first decodes the shape latent into coarse points, and then recovers the fine-grained details around each coarse point by deforming a 2D grid. To reconstruct the shape latent from partial observation, the diffusion network treats the partial observation as the conditional input and generates the shape latent via DDPM. Extensive experiments conducted on MVP, Completion3D, and KITTI quantitatively and qualitatively demonstrate the efficacy of PointLDM over the state-of-the-art shape completion approaches.

DeformingNet: Deforming Multiple Uniform 3D Priors for 3D Point Cloud Completion

Adaptive Recurrent Forward Network for Dense Point Cloud Completion

Point Cloud Completion Cascade Optimization Network Based on Feature Fusion

Dual-scale Point Cloud Completion Network Based on High-Frequency Feature Fusion

RFNet - Recurrent Forward Network for Dense Point Cloud Completion.

PMP-Net: Point Cloud Completion by Learning Multi-step Point Moving Paths

PMP-Net++: Point Cloud Completion by Transformer-Enhanced Multi-step Point Moving Paths

Point Projection Network: A Multi-View-Based Point Completion Network with Encoder-Decoder Architecture

Graph-Guided Deformation for Point Cloud Completion

DF-Net: Dynamic and Folding Network for 3D Point Cloud Completion

MFM-Net: Unpaired Shape Completion Network with Multi-stage Feature Matching.

Learning Geometric Transformation for Point Cloud Completion

DeformNet: Free-Form Deformation Network for 3D Shape Reconstruction from a Single Image

PF-Net: Point Fractal Network for 3D Point Cloud Completion

Learning 3D Shape Latent for Point Cloud Completion

CarvingNet: Point Cloud Completion by Stepwise Refining Multi-Resolution Features

Multistage Adaptive Point-Growth Network for Dense Point Cloud Completion

N-DPC: Dense 3D Point Cloud Completion Based on Improved Multi-Stage Network

ASHF-Net: Adaptive Sampling and Hierarchical Folding Network for Robust Point Cloud Completion

MLFT-Net: Point Cloud Completion Using MultiLevel Feature Transformer

FuNet: Multi-Feature Fusion for Point Cloud Completion Network