Abstract:Recent advances in 3D deep learning have shown that it is possible to train highly effective deep models for 3D shape generation, directly from 2D images. This is particularly interesting since the availability of 3D models is still limited compared to the massive amount of accessible 2D images, which is invaluable for training. The representation of 3D surfaces itself is a key factor for the quality and resolution of the 3D output. While explicit representations, such as point clouds and voxels, can span a wide range of shape variations, their resolutions are often limited. Mesh-based representations are more efficient but are limited by their ability to handle varying topologies. Implicit surfaces, however, can robustly handle complex shapes, topologies, and also provide flexible resolution control. We address the fundamental problem of learning implicit surfaces for shape inference without the need of 3D supervision. Despite their advantages, it remains nontrivial to (1) formulate a differentiable connection between implicit surfaces and their 2D renderings, which is needed for image-based supervision; and (2) ensure precise geometric properties and control, such as local smoothness. In particular, sampling implicit surfaces densely is also known to be a computationally demanding and very slow operation. To this end, we propose a novel ray-based field probing technique for efficient image-to-field supervision, as well as a general geometric regularizer for implicit surfaces, which provides natural shape priors in unconstrained regions. We demonstrate the effectiveness of our framework on the task of single-view image-based 3D shape digitization and show how we outperform state-of-the-art techniques both quantitatively and qualitatively.

PDE-Driven Implicit Reconstruction of 3D Object

Level-Set Based 3D Reconstruction Algorithm from Unorganized Data Cloud

Implicit Progressive-Iterative Approximation for Curve and Surface Reconstruction.

Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction

Topology-controllable Implicit Surface Reconstruction Based on Persistent Homology

Deep Implicit Moving Least-Squares Functions for 3D Reconstruction

Implicit Filtering for Learning Neural Signed Distance Functions from 3D Point Clouds

Edge Preserving Implicit Surface Representation of Point Clouds

3D Keypoint Estimation Using Implicit Representation Learning

Shrinking: Reconstruction of Parameterized Surfaces from Signed Distance Fields

Learning to Infer Implicit Surfaces without 3D Supervision

Gradient-SDF: A Semi-Implicit Surface Representation for 3D Reconstruction

Latent Partition Implicit with Surface Codes for 3D Representation

Automated Reconstruction of 3D Open Surfaces from Sparse Point Clouds

Learning Occupancy Function from Point Clouds for Surface Reconstruction

Shape As Points: A Differentiable Poisson Solver

3D Surface Reconstruction in the Wild by Deforming Shape Priors from Synthetic Data

Compact Implicit Surface Reconstruction Via Low-Rank Tensor Approximation

$p$-Poisson surface reconstruction in curl-free flow from point clouds

Neural-Singular-Hessian: Implicit Neural Representation of Unoriented Point Clouds by Enforcing Singular Hessian

Curvature Regularized Surface Reconstruction from Point Cloud