Abstract:Non-line-of-sight (NLOS) imaging is conducted to infer invisible scenes from indirect light on visible objects. The neural transient field (NeTF) was proposed for representing scenes as neural radiance fields in NLOS scenes. We propose NLOS neural implicit surface (NLOS-NeuS), which extends the NeTF to neural implicit surfaces with a signed distance function (SDF) for reconstructing three-dimensional surfaces in NLOS scenes. We introduce two constraints as loss functions for correctly learning an SDF to avoid non-zero level-set surfaces. We also introduce a lower bound constraint of an SDF based on the geometry of the first-returning photons. The experimental results indicate that these constraints are essential for learning a correct SDF in NLOS scenes. Compared with previous methods with discretized representation, NLOS-NeuS with the neural continuous representation enables us to reconstruct smooth surfaces while preserving fine details in NLOS scenes. To the best of our knowledge, this is the first study on neural implicit surfaces with volume rendering in NLOS scenes.

What problem does this paper attempt to address?

This paper attempts to address the problem of reconstructing 3D surfaces in non-line-of-sight (NLOS) scenarios. Specifically, the paper proposes a method called NLOS-NeuS, which extends the neural transient field (NeTF) to neural implicit surface representation, using the signed distance function (SDF) to reconstruct 3D surfaces in NLOS scenarios. The main goal of the paper is to overcome the limitations of existing methods in handling fine details and smooth surfaces, and to propose effective training losses to ensure the correct learning of SDF, thereby avoiding the appearance of non-zero level set surfaces. ### Main Contributions 1. **Proposing NLOS-NeuS**: This is the first study to use neural implicit surfaces with volumetric rendering in NLOS scenarios. 2. **Introducing new constraints and loss functions**: To correctly learn SDF, the paper introduces two main constraints, namely zero signed distance self-supervision and volumetric rendering weight constraint, to avoid the appearance of non-zero level set surfaces. 3. **Background rendering**: The paper also proposes a background network to handle background effects in real-world scenarios, such as inter-reflections or floor-reflected light. ### Problems Addressed - **Non-zero level set surfaces**: In NLOS settings, since the target object can only be observed from one side, traditional neural implicit surface methods are prone to generating non-zero level set surfaces. The paper addresses this issue by introducing specific loss functions. - **Reconstruction of fine details and smooth surfaces**: Existing discrete representation methods have limitations in handling fine details and smooth surfaces. NLOS-NeuS, through continuous neural implicit representation, can reconstruct smoother surfaces while preserving fine details. - **Background effects**: Background effects in real-world scenarios (such as inter-reflections or floor-reflected light) can affect reconstruction results. The paper effectively handles these background effects through a background network. ### Experimental Results Experimental results show that NLOS-NeuS outperforms existing methods in terms of reconstruction quality and detail preservation, especially when dealing with complex geometric structures. For example, in reconstructing complex structures such as rabbit ears, Lucy's wings, and Indonesian head statues, NLOS-NeuS is better able to capture these details. Overall, this paper makes significant contributions to the field of 3D surface reconstruction in NLOS scenarios, significantly improving reconstruction quality and accuracy by introducing new techniques and methods.

NLOS-NeuS: Non-line-of-sight Neural Implicit Surface

Non-line-of-Sight Imaging via Neural Transient Fields

NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction

Towards Non-Line-of-Sight Photography

Occlusion Fields: An Implicit Representation for Non-Line-of-Sight Surface Reconstruction

Anti-Aliased Neural Implicit Surfaces with Encoding Level of Detail

Deep Non-line-of-sight Imaging from Under-scanning Measurements.

Learned feature embeddings for non-line-of-sight imaging and recognition

Neural Unsigned Distance Fields for Implicit Function Learning

Dynamic-excitation-based Steady-State Non-Line-of-sight Imaging Via Multi-Branch Convolutional Neural Network

KiloNeuS: A Versatile Neural Implicit Surface Representation for Real-Time Rendering

From Transparent to Opaque: Rethinking Neural Implicit Surfaces with $α$-NeuS

NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild

Depth-NeuS: Neural Implicit Surfaces Learning for Multi-view Reconstruction Based on Depth Information Optimization

Non-Line-of-Sight Imaging with Signal Superresolution Network

NeAT: Learning Neural Implicit Surfaces with Arbitrary Topologies from Multi-view Images

Physics to the Rescue: Deep Non-line-of-sight Reconstruction for High-speed Imaging

Recent Advances on Non-Line-of-Sight Imaging: Conventional Physical Models, Deep Learning, and New Scenes

Non-line-of-sight Imaging with Arbitrary Illumination and Detection Pattern

Generalizable Non-Line-of-Sight Imaging with Learnable Physical Priors

Multi-scale Iterative Model-guided Unfolding Network for NLOS Reconstruction.