Abstract:Recent history has seen a tremendous growth of work exploring implicit representations of geometry and radiance, popularized through Neural Radiance Fields (NeRF). Such works are fundamentally based on a (implicit) volumetric representation of occupancy, allowing them to model diverse scene structure including translucent objects and atmospheric obscurants. But because the vast majority of real-world scenes are composed of well-defined surfaces, we introduce a surface analog of such implicit models called Neural Reflectance Surfaces (NeRS). NeRS learns a neural shape representation of a closed surface that is diffeomorphic to a sphere, guaranteeing water-tight reconstructions. Even more importantly, surface parameterizations allow NeRS to learn (neural) bidirectional surface reflectance functions (BRDFs) that factorize view-dependent appearance into environmental illumination, diffuse color (albedo), and specular "shininess." Finally, rather than illustrating our results on synthetic scenes or controlled in-the-lab capture, we assemble a novel dataset of multi-view images from online marketplaces for selling goods. Such "in-the-wild" multi-view image sets pose a number of challenges, including a small number of views with unknown/rough camera estimates. We demonstrate that surface-based neural reconstructions enable learning from such data, outperforming volumetric neural rendering-based reconstructions. We hope that NeRS serves as a first step toward building scalable, high-quality libraries of real-world shape, materials, and illumination. The project page with code and video visualizations can be found at <a class="link-external link-https" href="https://jasonyzhang.com/ners" rel="external noopener nofollow">this https URL</a>.

NeuralTO: Neural Reconstruction and View Synthesis of Translucent Objects

ReN Human: Learning Relightable Neural Implicit Surfaces for Animatable Human Rendering

Seeing Through the Glass: Neural 3D Reconstruction of Object Inside a Transparent Container

NeTO:Neural Reconstruction of Transparent Objects with Self-Occlusion Aware Refraction-Tracing

NeTO: Neural Reconstruction of Transparent Objects with Self-Occlusion Aware Refraction-Tracing.

Looking Through the Glass: Neural Surface Reconstruction Against High Specular Reflections

From Transparent to Opaque: Rethinking Neural Implicit Surfaces with $α$-NeuS

Dynamic Multi-View Scene Reconstruction Using Neural Implicit Surface

Hybrid Mesh-neural Representation for 3D Transparent Object Reconstruction

Neural 3D Reconstruction in the Wild

3D Reconstruction and New View Synthesis of Indoor Environments based on a Dual Neural Radiance Field

NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild

Neural Rays for Occlusion-aware Image-based Rendering

NeuralRoom: Geometry-Constrained Neural Implicit Surfaces for Indoor Scene Reconstruction.

NeuralRoom

Neural 3D reconstruction from sparse views using geometric priors

Neural Implicit 3D Reconstruction with Double Supervision

UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction

Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflection

Res-NeuS: Deep Residuals and Neural Implicit Surface Learning for Multi-View Reconstruction

Efficient Implicit Neural Reconstruction Using LiDAR