Abstract:Deep neural networks (DNNs) are widely applied for nowadays 3D surface reconstruction tasks and such methods can be further divided into two categories, which respectively warp templates explicitly by moving vertices or represent 3D surfaces implicitly as signed or unsigned distance functions. Taking advantage of both advanced explicit learning process and powerful representation ability of implicit functions, we propose a novel 3D representation method, Neural Vector Fields (NVF). It not only adopts the explicit learning process to manipulate meshes directly, but also leverages the implicit representation of unsigned distance functions (UDFs) to break the barriers in resolution and topology. Specifically, our method first predicts the displacements from queries towards the surface and models the shapes as \textit{Vector Fields}. Rather than relying on network differentiation to obtain direction fields as most existing UDF-based methods, the produced vector fields encode the distance and direction fields both and mitigate the ambiguity at "ridge" points, such that the calculation of direction fields is straightforward and differentiation-free. The differentiation-free characteristic enables us to further learn a shape codebook via Vector Quantization, which encodes the cross-object priors, accelerates the training procedure, and boosts model generalization on cross-category reconstruction. The extensive experiments on surface reconstruction benchmarks indicate that our method outperforms those state-of-the-art methods in different evaluation scenarios including watertight vs non-watertight shapes, category-specific vs category-agnostic reconstruction, category-unseen reconstruction, and cross-domain reconstruction. Our code is released at <a class="link-external link-https" href="https://github.com/Wi-sc/NVF" rel="external noopener nofollow">this https URL</a>.

Multiple Level of Details Neural Implicit Surface Representation for Unconstrained Viewpoint Rendering

Anti-Aliased Neural Implicit Surfaces with Encoding Level of Detail

Point-Based Neural Scene Rendering for Street Views

Res-NeuS: Deep Residuals and Neural Implicit Surface Learning for Multi-View Reconstruction

ReN Human: Learning Relightable Neural Implicit Surfaces for Animatable Human Rendering

Multiscale Representation for Real-Time Anti-Aliasing Neural Rendering

Neural Global Illumination Via Superposed Deformable Feature Fields

Recovering Fine Details for Neural Implicit Surface Reconstruction

Dynamic Multi-View Scene Reconstruction Using Neural Implicit Surface

Depth Guided Fast Rendering Neural Radiance Fields

Voxel-Mesh Hybrid Representation for Real-Time View Synthesis

VQ-NeRF: Vector Quantization Enhances Implicit Neural Representations

HR-NeuS: Recovering High-Frequency Surface Geometry via Neural Implicit Surfaces

Neuralangelo: High-Fidelity Neural Surface Reconstruction

Neural Vector Fields: Implicit Representation by Explicit Learning

Anisotropic Neural Representation Learning for High-Quality Neural Rendering

Explicifying Neural Implicit Fields for Efficient Dynamic Human Avatar Modeling via a Neural Explicit Surface

Edge-aware Neural Implicit Surface Reconstruction

Parameterization-driven Neural Surface Reconstruction for Object-oriented Editing in Neural Rendering

VET: Visual Error Tomography for Point Cloud Completion and High-Quality Neural Rendering

Fine-detailed Neural Indoor Scene Reconstruction using multi-level importance sampling and multi-view consistency