Abstract:We introduce Omni-LOS, a neural computational imaging method for conducting holistic shape reconstruction (HSR) of complex objects utilizing a Single-Photon Avalanche Diode (SPAD)-based time-of-flight sensor. As illustrated in Fig. 1, our method enables new capabilities to reconstruct near-$360^\circ$ surrounding geometry of an object from a single scan spot. In such a scenario, traditional line-of-sight (LOS) imaging methods only see the front part of the object and typically fail to recover the occluded back regions. Inspired by recent advances of non-line-of-sight (NLOS) imaging techniques which have demonstrated great power to reconstruct occluded objects, Omni-LOS marries LOS and NLOS together, leveraging their complementary advantages to jointly recover the holistic shape of the object from a single scan position. The core of our method is to put the object nearby diffuse walls and augment the LOS scan in the front view with the NLOS scans from the surrounding walls, which serve as virtual ``mirrors'' to trap lights toward the object. Instead of separately recovering the LOS and NLOS signals, we adopt an implicit neural network to represent the object, analogous to NeRF and NeTF. While transients are measured along straight rays in LOS but over the spherical wavefronts in NLOS, we derive differentiable ray propagation models to simultaneously model both types of transient measurements so that the NLOS reconstruction also takes into account the direct LOS measurements and vice versa. We further develop a proof-of-concept Omni-LOS hardware prototype for real-world validation. Comprehensive experiments on various wall settings demonstrate that Omni-LOS successfully resolves shape ambiguities caused by occlusions, achieves high-fidelity 3D scan quality, and manages to recover objects of various scales and complexity.

HOLI-1-to-3: Transient-Enhanced Holistic Image-to-3D Generation

Omni-Line-of-Sight Imaging for Holistic Shape Reconstruction

In-Hand 3D Object Reconstruction from a Monocular RGB Video

One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization

Dynamic-excitation-based Steady-State Non-Line-of-sight Imaging Via Multi-Branch Convolutional Neural Network

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

3-D Data Merging Using Holoimage

Three-Dimensional Data Merging Using Holoimage

Plug-and-Play Algorithms for Dynamic Non-line-of-sight Imaging

Towards Non-Line-of-Sight Photography

Three-Dimensional Holographic Reconstruction from Computational Tomography Images

Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation

Unsupervised Deep Learning Enables 3D Imaging for Single‐Shot Incoherent Holography

Holo-Relighting: Controllable Volumetric Portrait Relighting from a Single Image

Photon-Efficient Non-Line-of-Sight Imaging

Rapid Extraction of 3d Regions of Interest from Digital Holograms

Holistic++ Scene Understanding: Single-View 3D Holistic Scene Parsing and Human Pose Estimation With Human-Object Interaction and Physical Commonsense

Holistic 3D Scene Understanding from a Single Image with Implicit Representation.

Deep Non-line-of-sight Imaging from Under-scanning Measurements.

Iterative Projection Meets Sparsity Regularization: Towards Practical Single-Shot Quantitative Phase Imaging with In-Line Holography

One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion