Abstract:The maturity of depth sensors and laser scanning techniques has enabled the convenient acquisition of 3D dynamic point clouds—one natural representation of 3D objects/scenes in motion, leading to a wide range of applications such as immersive tele-presence, autonomous driving, augmented and virtual reality. Nevertheless, dynamic point clouds usually exhibit holes of missing data, thus inpainting is crucial to the subsequent rendering or downstream understanding tasks. Dynamic point cloud inpainting has been largely overlooked so far, which is also quite challenging due to the irregular sampling patterns both in the spatial domain and temporal domain. To this end, we propose an efficient dynamic point cloud inpainting method based on a learnable spatial-temporal graph representation, exploiting both the second-order inter-frame coherence and the intra-frame self-similarity. The key is the second-order inter-frame coherence that enforces the consistent flow in 3D motion over time, for which we search the temporal correspondence in consecutive frames for the same underlying surface by the point-to-plane distance and represent the correlation between them via temporal edge weights in the graph. Based on the second-order inter-frame coherence and intra-frame self-similarity, we formulate dynamic point cloud inpainting as a joint optimization problem of the desired point cloud and underlying spatial-temporal graph, which is regularized by consistency in the temporal edge weights and smoothness in the spatial domain. We analyze and reformulate the optimization, leading to an efficient alternating minimization algorithm. Experimental results show that the proposed approach outperforms several competing methods significantly, both on synthetic holes and real holes.

Point Cloud Attribute Inpainting in Graph Spectral Domain.

Local Frequency Interpretation and Non-Local Self-Similarity on Graph for Point Cloud Inpainting

Dynamic Point Cloud Inpainting Via Spatial-Temporal Graph Learning

3D Dynamic Point Cloud Inpainting Via Temporal Consistency on Graphs

Terrain Point Cloud Inpainting via Signal Decomposition

Point Cloud Inpainting with Normal-Based Feature Matching

PLGP: Point Cloud Inpainting by Patch-Based Local Geometric Propagating

Hole Inpainting Algorithm for Half-organized Point Cloud Obtained by Structured-light Section System

Iterative Removal of G-PCC Attribute Compression Artifacts Based on a Graph Neural Network

3D Point Cloud Attribute Compression Using Diffusion-based Texture-aware Intra Prediction

Feature Preserving and Uniformity-controllable Point Cloud Simplification on Graph

PUI-Net: A Point Cloud Upsampling and Inpainting Network

A Hybrid Compression Framework for Color Attributes of Static 3D Point Clouds

Efficient Method For Geometric Attribute Estimation

Point Clouds Attribute Compression Using Data-Adaptive Intra Prediction

Holes Filling of Scattered Point Cloud Based on Simplification

Attribute Artifacts Removal for Geometry-based Point Cloud Compression

Texture-Guided Graph Transform Optimization for Point Cloud Attribute Compression

Rate-Distortion Optimized Graph for Point Cloud Attribute Coding

PointDreamer: Zero-shot 3D Textured Mesh Reconstruction from Colored Point Cloud by 2D Inpainting

Unsupervised Learning of Geometric Sampling Invariant Representations for 3D Point Clouds