Abstract:Neural implicit representations have emerged as a powerful paradigm for 3D reconstruction. However, despite their success, existing methods fail to capture fine geometric details and thin structures, especially in scenarios where only sparse RGB views of the objects of interest are available. We hypothesize that current methods for learning neural implicit representations from RGB or RGBD images produce 3D surfaces with missing parts and details because they only rely on 0-order differential properties, i.e. the 3D surface points and their projections, as supervisory signals. Such properties, however, do not capture the local 3D geometry around the points and also ignore the interactions between points. This paper demonstrates that training neural representations with first-order differential properties, i.e. surface normals, leads to highly accurate 3D surface reconstruction even in situations where only as few as two RGB (front and back) images are available. Given multiview RGB images of an object of interest, we first compute the approximate surface normals in the image space using the gradient of the depth maps produced using an off-the-shelf monocular depth estimator such as Depth Anything model. An implicit surface regressor is then trained using a loss function that enforces the first-order differential properties of the regressed surface to match those estimated from Depth Anything. Our extensive experiments on a wide range of real and synthetic datasets show that the proposed method achieves an unprecedented level of reconstruction accuracy even when using as few as two RGB views. The detailed ablation study also demonstrates that normal-based supervision plays a key role in this significant improvement in performance, enabling the 3D reconstruction of intricate geometric details and thin structures that were previously challenging to capture.

Edge-aware Neural Implicit Surface Reconstruction

Fine-detailed Neural Indoor Scene Reconstruction using multi-level importance sampling and multi-view consistency

Recovering Fine Details for Neural Implicit Surface Reconstruction

Three-Dimensional Reconstruction of Indoor Scenes Based on Implicit Neural Representation

NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction

Res-NeuS: Deep Residuals and Neural Implicit Surface Learning for Multi-View Reconstruction

Efficient Implicit Neural Reconstruction Using LiDAR

Error-aware Sampling in Adaptive Shells for Neural Surface Reconstruction

Edge Preserving Implicit Surface Representation of Point Clouds

HR-NeuS: Recovering High-Frequency Surface Geometry via Neural Implicit Surfaces

Autonomous Implicit Indoor Scene Reconstruction with Frontier Exploration

Neural 3D Reconstruction in the Wild

NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting Guidance

3D Neural Edge Reconstruction

Normal-guided Detail-Preserving Neural Implicit Functions for High-Fidelity 3D Surface Reconstruction

Neural Kernel Surface Reconstruction

Neural 3D Scene Reconstruction from Multiple 2D Images without 3D Supervision

NeurAR: Neural Uncertainty for Autonomous 3D Reconstruction With Implicit Neural Representations

ReN Human: Learning Relightable Neural Implicit Surfaces for Animatable Human Rendering

Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflection

Enhancing Surface Neural Implicits with Curvature-Guided Sampling and Uncertainty-Augmented Representations