Abstract:Recent advances in 3D deep learning have shown that it is possible to train highly effective deep models for 3D shape generation, directly from 2D images. This is particularly interesting since the availability of 3D models is still limited compared to the massive amount of accessible 2D images, which is invaluable for training. The representation of 3D surfaces itself is a key factor for the quality and resolution of the 3D output. While explicit representations, such as point clouds and voxels, can span a wide range of shape variations, their resolutions are often limited. Mesh-based representations are more efficient but are limited by their ability to handle varying topologies. Implicit surfaces, however, can robustly handle complex shapes, topologies, and also provide flexible resolution control. We address the fundamental problem of learning implicit surfaces for shape inference without the need of 3D supervision. Despite their advantages, it remains nontrivial to (1) formulate a differentiable connection between implicit surfaces and their 2D renderings, which is needed for image-based supervision; and (2) ensure precise geometric properties and control, such as local smoothness. In particular, sampling implicit surfaces densely is also known to be a computationally demanding and very slow operation. To this end, we propose a novel ray-based field probing technique for efficient image-to-field supervision, as well as a general geometric regularizer for implicit surfaces, which provides natural shape priors in unconstrained regions. We demonstrate the effectiveness of our framework on the task of single-view image-based 3D shape digitization and show how we outperform state-of-the-art techniques both quantitatively and qualitatively.

Learning to Infer Implicit Surfaces without 3D Supervision

LISR: Learning Linear 3D Implicit Surface Representation Using Compactly Supported Radial Basis Functions

Deep Implicit Moving Least-Squares Functions for 3D Reconstruction

Learning Shape Primitives Via Implicit Convexity Regularization

Local Deep Implicit Functions for 3D Shape

Inferring 3D Occupancy Fields Through Implicit Reasoning on Silhouette Images

NeAT: Learning Neural Implicit Surfaces with Arbitrary Topologies from Multi-view Images

Geometry-Guided Neural Implicit Surface Reconstruction

Neural Implicit 3D Reconstruction with Double Supervision

Deep Optimized Priors for 3D Shape Modeling and Reconstruction

HelixSurf: A Robust and Efficient Neural Implicit Surface Learning of Indoor Scenes with Iterative Intertwined Regularization

NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction

Neural Implicit Surface Reconstruction of Freehand 3D Ultrasound Volume with Geometric Constraints

Joint stereo 3D object detection and implicit surface reconstruction

Self-Supervised Implicit 3D Reconstruction via RGB-D Scans.

Enhancing Surface Neural Implicits with Curvature-Guided Sampling and Uncertainty-Augmented Representations

Towards Unbiased Volume Rendering of Neural Implicit Surfaces with Geometry Priors

Geo-Neus: Geometry-Consistent Neural Implicit Surfaces Learning for Multi-view Reconstruction

Deep Marching Cubes: Learning Explicit Surface Representations.

Deep Learning 3d Shape Surfaces Using Geometry Images

Depth-NeuS: Neural Implicit Surfaces Learning for Multi-view Reconstruction Based on Depth Information Optimization