Abstract:Object-centric surface reconstruction from multi-view images is crucial in creating editable digital assets for AR/VR. Due to the lack of geometric constraints, existing methods, e.g., NeuS necessitate annotating the object masks to reconstruct compact surfaces in mesh processing. Mask annotation, however, incurs considerable labor costs due to its cumbersome nature. This paper presents Hi-NeuS, a novel rendering-based framework for neural implicit surface reconstruction, aiming to recover compact and precise surfaces without multi-view object masks. Our key insight is that the overlapping regions in the object-centric views naturally highlight the object of interest as the camera orbits around objects. The object of interest can be specified by estimating the distribution of the rendering weights accumulated from multiple views, which implicitly identifies the surface that a user intends to capture. This inspires us to design a geometric refinement approach, which takes multi-view rendering weights to guide the signed distance functions (SDF) of neural surfaces in a self-supervised manner. Specifically, it retains these weights to resample a pseudo surface based on their distribution. This facilitates the alignment of the SDF to the object of interest. We then regularize the SDF's bias for geometric consistency. Moreover, we propose to use unmasked Chamfer Distance(CD) to measure the extracted mesh without post-processing for more precise evaluation. Our approach has been validated through NeuS and its variant Neuralangelo, demonstrating its adaptability across different NeuS backbones. Extensive benchmark on the DTU dataset shows that our method reduces surface noise by about 20%, and improves the unmasked CD by around 30%, achieving better surface details. The superiority of Hi-NeuS is further validated on BlendedMVS and handheld camera captures for content creation.

Enhancing single-view 3D mesh reconstruction with the aid of implicit surface learning

Saliency Guided Subdivision for Single-View Mesh Reconstruction

Geo-Neus: Geometry-Consistent Neural Implicit Surfaces Learning for Multi-view Reconstruction

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Sat-Mesh: Learning Neural Implicit Surfaces for Multi-View Satellite Reconstruction

Vis2Mesh: Efficient Mesh Reconstruction from Unstructured Point Clouds of Large Scenes with Learned Virtual View Visibility

Single-view 3D Mesh Reconstruction for Seen and Unseen Categories

Sur2f: A Hybrid Representation for High-Quality and Efficient Surface Reconstruction from Multi-view Images

Multiview Textured Mesh Recovery by Differentiable Rendering

Hybrid Mesh-neural Representation for 3D Transparent Object Reconstruction

2L3: Lifting Imperfect Generated 2D Images into Accurate 3D

LIST: Learning Implicitly from Spatial Transformers for Single-View 3D Reconstruction

Hybrid-CSR: Coupling Explicit and Implicit Shape Representation for Cortical Surface Reconstruction

Dynamic Multi-View Scene Reconstruction Using Neural Implicit Surface

Deep Single-View 3D Object Reconstruction with Visual Hull Embedding

High-Fidelity Mask-free Neural Surface Reconstruction for Virtual Reality

Coupling Explicit and Implicit Surface Representations for Generative 3D Modeling

Learning to Infer Implicit Surfaces without 3D Supervision

Deep Mesh Reconstruction from Single RGB Images via Topology Modification Networks

NeuSurf: On-Surface Priors for Neural Surface Reconstruction from Sparse Input Views

Deep Hybrid Self-Prior for Full 3D Mesh Generation