Abstract:By formulating the data generation as a sequence procedure of denoising autoencoding, diffusion models have achieved superior in-painting performance on image data and beyond. Nevertheless, it is not trivial when capitalizing on diffusion models to generate missing 3D points. The difficulty originates from the intrinsic structure where 3D point cloud is a set of unordered and irregular coordinates. That motivates us to delve into the 3D structural information for designing point cloud encoder-decoder and shape latent generator, to precisely formulate the latent distribution of the complete point cloud and partial observation. In this paper, we propose Point cloud completion with Latent Diffusion Models (PointLDM), a new approach that leverages the conditional denoising diffusion probabilistic modeling (DDPM) in the 3D latent space for shape reconstruction. The architecture of PointLDM consists of a transformer-based variational auto-encoder (VAE) to model the complete shape latent, and a diffusion network for shape latent prediction. The encoder of VAE exploits both of global shape latent and local point features in shape distribution learning. With the learnt shape latent, the decoder first decodes the shape latent into coarse points, and then recovers the fine-grained details around each coarse point by deforming a 2D grid. To reconstruct the shape latent from partial observation, the diffusion network treats the partial observation as the conditional input and generates the shape latent via DDPM. Extensive experiments conducted on MVP, Completion3D, and KITTI quantitatively and qualitatively demonstrate the efficacy of PointLDM over the state-of-the-art shape completion approaches.

A Geometry Aware Diffusion Model for 3D Point Cloud Generation

Diffusion Probabilistic Models for 3D Point Cloud Generation

Deformable 3D Shape Diffusion Model

Controllable Mesh Generation Through Sparse Latent Point Diffusion Models.

Diff-pcg: diffusion point cloud generation conditioned on continuous normalizing flow

Locally Attentional SDF Diffusion for Controllable 3D Shape Generation

3D Shape Generation and Completion through Point-Voxel Diffusion

Learning 3D Shape Latent for Point Cloud Completion

Part-aware Shape Generation with Latent 3D Diffusion of Neural Voxel Fields

Neural Point Cloud Diffusion for Disentangled 3D Shape and Appearance Generation

Topology-Aware Latent Diffusion for 3D Shape Generation

Sketch and Text Guided Diffusion Model for Colored Point Cloud Generation

Patch-Wise Point Cloud Generation: A Divide-and-Conquer Approach

Efficient and Scalable Point Cloud Generation with Sparse Point-Voxel Diffusion Models

GECCO: Geometrically-Conditioned Point Diffusion Models

Fast Training of Diffusion Transformer with Extreme Masking for 3D Point Clouds Generation

Enhancing Diffusion-based Point Cloud Generation with Smoothness Constraint

GaussianAnything: Interactive Point Cloud Latent Diffusion for 3D Generation

Dual adversarial model: Exploring low-dimensional space features for point clouds generating and completing

Point Cloud Resampling with Learnable Heat Diffusion

BuilDiff: 3D Building Shape Generation using Single-Image Conditional Point Cloud Diffusion Models