Abstract:Computational approach to imaging around the corner, or non-line-of-sight (NLOS) imaging, is becoming a reality thanks to major advances in imaging hardware and reconstruction algorithms. A recent development towards practical NLOS imaging, Nam et al. demonstrated a high-speed non-confocal imaging system that operates at 5Hz, 100x faster than the prior art. This enormous gain in acquisition rate, however, necessitates numerous approximations in light transport, breaking many existing NLOS reconstruction methods that assume an idealized image formation model. To bridge the gap, we present a novel deep model that incorporates the complementary physics priors of wave propagation and volume rendering into a neural network for high-quality and robust NLOS reconstruction. This orchestrated design regularizes the solution space by relaxing the image formation model, resulting in a deep model that generalizes well on real captures despite being exclusively trained on synthetic data. Further, we devise a unified learning framework that enables our model to be flexibly trained using diverse supervision signals, including target intensity images or even raw NLOS transient measurements. Once trained, our model renders both intensity and depth images at inference time in a single forward pass, capable of processing more than 5 captures per second on a high-end GPU. Through extensive qualitative and quantitative experiments, we show that our method outperforms prior physics and learning based approaches on both synthetic and real measurements. We anticipate that our method along with the fast capturing system will accelerate future development of NLOS imaging for real world applications that require high-speed imaging.

Fast Full-wave Microwave Imaging with Physics Embedded Deep Neural Network

Optimal Phase Mask Design for Coherent Modulation Imaging by Deep Learning

Universal and High-Fidelity Resolution Extending for Fluorescence Microscopy Using a Single-Training Physics-Informed Sparse Neural Network

Wavefront Reconstruction Based on Deep Transfer Learning for Microscopy.

The Integration of Neural Network and Physical Reconstruction Model for Fourier Ptychographic Microscopy

Deep Learning with Physics-embedded Neural Network for Full Waveform Ultrasonic Brain Imaging

Fast Near-Field Frequency-Diverse Computational Imaging Based on End-to-End Deep-Learning Network

Single Plane-Wave Imaging using Physics-Based Deep Learning

Physics to the Rescue: Deep Non-line-of-sight Reconstruction for High-speed Imaging

Physics Embedded Deep Neural Network for Solving Full-Wave Inverse Scattering Problems

MiPhDUO: Microwave imaging via physics‐informed deep unrolled optimization

Microwave Imaging by Deep Learning Network: Feasibility and Training Method

Model-Based Explainable Deep Learning for Light-Field Microscopy Imaging

Dynamic-excitation-based Steady-State Non-Line-of-sight Imaging Via Multi-Branch Convolutional Neural Network

Physics-based Iterative Projection Complex Neural Network for Phase Retrieval in Lensless Microscopy Imaging

Fourier Ptychography Based on Deep Learning

MDLI-Net: Model-Driven Learning Imaging Network for High-Resolution Microwave Imaging with Large Rotating Angle and Sparse Sampling

Ultracompact Deep Neural Network for Ultrafast Optical Property Extraction in Spatial Frequency Domain Imaging (SFDI)

WaveY-Net: Physics-augmented deep learning for high-speed electromagnetic simulation and optimization

Fast VGG: An Advanced Pre-Trained Deep Learning Framework for Multi-Layered Composite NDE via Multifrequency Near-Field Microwave Imaging

DeepUCT: Complex cascaded deep learning network for improved ultrasound tomography