Abstract:It is challenging to reconstruct 3D point clouds in unseen classes from single 2D images. Instead of object-centered coordinate system, current methods generalized global priors learned in seen classes to reconstruct 3D shapes from unseen classes in viewer-centered coordinate system. However, the reconstruction accuracy and interpretability are still eager to get improved. To resolve this issue, we introduce to learn local pattern modularization for reconstructing 3D shapes in unseen classes, which achieves both good generalization ability and high reconstruction accuracy. Our insight is to learn a local prior which is class-agnostic and easy to generalize in object-centered coordinate system. Specifically, the local prior is learned via a process of learning and customizing local pattern modularization in seen classes. During this process, we first learn a set of patterns in local regions, which is the basis in the object-centered coordinate system to represent an arbitrary region on shapes across different classes. Then, we modularize each region on an initially reconstructed shape using the learned local patterns. Based on that, we customize the local pattern modularization using the input image by refining the reconstruction with more details. Our method enables to reconstruct high fidelity point clouds from unseen classes in object-centered coordinate system without requiring a large number of patterns or any additional information, such as segmentation supervision or camera poses. Our experimental results under widely used benchmarks show that our method achieves the state-of-the-art reconstruction accuracy for shapes from unseen classes. The code is available at <a class="link-external link-https" href="https://github.com/chenchao15/Unseen" rel="external noopener nofollow">this https URL</a>.

Learnable Chamfer Distance for Point Cloud Reconstruction

Learning to Measure the Point Cloud Reconstruction Loss in a Representation Space

Learning to Train a Point Cloud Reconstruction Network Without Matching

RFNet - Recurrent Forward Network for Dense Point Cloud Completion.

A Point Matching Strategy of 3D Loss Function for Single RGB Images Deep Mesh Reconstruction

Oriented and Directional Chamfer Distance Losses for 3D Object Reconstruction from a Single Image

Visual Enhanced 3D Point Cloud Reconstruction from A Single Image

Learning Density Regulated and Multi-View Consistent Unsigned Distance Fields.

Visual enhancement of single-view 3D point cloud reconstruction

3D Reconstruction and Semantic Segmentation Method Combining PointNet and 3D-Lmnet from Single Image

A Visualization-aware Loss for Point Cloud Comparison

Hyperbolic Chamfer Distance for Point Cloud Completion and Beyond

Loss Distillation via Gradient Matching for Point Cloud Completion with Weighted Chamfer Distance

Point Distance Mask Attention Network for Learning on Point Cloud

Dense Point Cloud Reconstruction by Shape and Pose Features Learning

Learning Local Pattern Modularization for Point Cloud Reconstruction from Unseen Classes

DiffCD: A Symmetric Differentiable Chamfer Distance for Neural Implicit Surface Fitting

CD2: Fine-grained 3D Mesh Reconstruction with Twice Chamfer Distance

CAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimization

ToThePoint: Efficient Contrastive Learning of 3D Point Clouds Via Recycling