Abstract:We introduce Omni-LOS, a neural computational imaging method for conducting holistic shape reconstruction (HSR) of complex objects utilizing a Single-Photon Avalanche Diode (SPAD)-based time-of-flight sensor. As illustrated in Fig. 1, our method enables new capabilities to reconstruct near-$360^\circ$ surrounding geometry of an object from a single scan spot. In such a scenario, traditional line-of-sight (LOS) imaging methods only see the front part of the object and typically fail to recover the occluded back regions. Inspired by recent advances of non-line-of-sight (NLOS) imaging techniques which have demonstrated great power to reconstruct occluded objects, Omni-LOS marries LOS and NLOS together, leveraging their complementary advantages to jointly recover the holistic shape of the object from a single scan position. The core of our method is to put the object nearby diffuse walls and augment the LOS scan in the front view with the NLOS scans from the surrounding walls, which serve as virtual ``mirrors'' to trap lights toward the object. Instead of separately recovering the LOS and NLOS signals, we adopt an implicit neural network to represent the object, analogous to NeRF and NeTF. While transients are measured along straight rays in LOS but over the spherical wavefronts in NLOS, we derive differentiable ray propagation models to simultaneously model both types of transient measurements so that the NLOS reconstruction also takes into account the direct LOS measurements and vice versa. We further develop a proof-of-concept Omni-LOS hardware prototype for real-world validation. Comprehensive experiments on various wall settings demonstrate that Omni-LOS successfully resolves shape ambiguities caused by occlusions, achieves high-fidelity 3D scan quality, and manages to recover objects of various scales and complexity.

Recent Progress of Non-Line-of-sight Imaging Reconstruction Algorithms in Typical Imaging Modalities

Recent Advances on Non-Line-of-Sight Imaging: Conventional Physical Models, Deep Learning, and New Scenes

Dynamic-excitation-based Steady-State Non-Line-of-sight Imaging Via Multi-Branch Convolutional Neural Network

Non-Line-of-Sight Imaging with Signal Superresolution Network

Research Advances on Non-Line-of-Sight Imaging Technology

Physics to the Rescue: Deep Non-line-of-sight Reconstruction for High-speed Imaging

Long-Range Non-Line-of-Sight Imaging Based on Projected Images from Multiple Light Fields

Non-line-of-sight Imaging with Arbitrary Illumination and Detection Pattern

Non-line-of-sight reconstruction via structure sparsity regularization

Dynamic Non-Line-of-sight Imaging System Based on the Optimization of Point Spread Functions.

Fast Non-Line-of-Sight Imaging With Hybrid Super-Resolution Network Over 18 m

Domain Reduction Strategy for Non Line of Sight Imaging

45.2: High‐performance Non‐Line‐of‐Sight imaging based on deep learning

Plug-and-Play Algorithms for Dynamic Non-line-of-sight Imaging

High-resolution non-confocal non-line-of-sight imaging based on spherical-slice transform from spatial and temporal frequency to space and time

An Efficient Way for Active None-Line-of-Sight: End-to-End Learned Compressed NLOS Imaging.

Omni-Line-of-Sight Imaging for Holistic Shape Reconstruction

Non-line-of-Sight Imaging via Neural Transient Fields

Non-line-of-sight imaging with adaptive artifact cancellation

Improved Algorithm of Non-Line-of-sight Imaging Based on the Bayesian Statistics

Non-line-of-sight imaging based on Archimedean spiral scanning