Abstract:Single-photon avalanche diodes (SPADs) are advanced sensors capable of detecting individual photons and recording their arrival times with picosecond resolution using time-correlated Single-Photon Counting detection techniques. They are used in various applications, such as LiDAR, and can capture high-speed sequences of binary single-photon images, offering great potential for reconstructing 3D environments with high motion dynamics. To complement single-photon data, they are often paired with conventional passive cameras, which capture high-resolution (HR) intensity images at a lower frame rate. However, 3D reconstruction from SPAD data faces challenges. Aggregating multiple binary measurements improves precision and reduces noise but can cause motion blur in dynamic scenes. Additionally, SPAD arrays often have lower resolution than passive cameras. To address these issues, we propose a novel computational imaging algorithm to improve the 3D reconstruction of moving scenes from SPAD data by addressing the motion blur and increasing the native spatial resolution. We adopt a plug-and-play approach within an optimization scheme alternating between guided video super-resolution of the 3D scene, and precise image realignment using optical flow. Experiments on synthetic data show significantly improved image resolutions across various signal-to-noise ratios and photon levels. We validate our method using real-world SPAD measurements on three practical situations with dynamic objects. First on fast-moving scenes in laboratory conditions at short range; second very low resolution imaging of people with a consumer-grade SPAD sensor from STMicroelectronics; and finally, HR imaging of people walking outdoors in daylight at a range of 325 meters under eye-safe illumination conditions using a short-wave infrared SPAD camera. These results demonstrate the robustness and versatility of our approach.

Subpixel Three Dimensional Imaging Using Two Dimensional Optical Encoding with a Downscaled Avalanche Photodiode Array

Subpixel Three-Dimensional Laser Imaging with a Downscaled Avalanche Photodiode Array Using Code Division Multiple Access

3D Framing Camera System Based on Sample Cumulation Method

Pixel Multiplexing Technique For Real-Time Three-Dimensional-Imaging Laser Detection And Ranging System Using Four Linear-Mode Avalanche Photodiodes

Non-Line-of-Sight Three-Dimensional Imaging with a Single-Pixel Camera

Optical OCDMA Coding and 3D Imaging Technique for Non-Scanning Full-Waveform LiDAR System.

Deep Learning Based Single-Photon 3D Imaging with Multiple Returns

A Plug-and-Play Algorithm for 3D Video Super-Resolution of Single-Photon LiDAR data

Single-photon depth imaging using a photodetector with a wire-bonding quenching resistor

Single photon imaging based on a photon driven sparse sampling

PCA-based real-time single-photon 3D imaging method

High-resolution multi-spectral snapshot 3D imaging with a SPAD array camera

Single-Photon 3D Imaging with Equi-Depth Photon Histograms

Development of a near-infrared single-photon 3D imaging LiDAR based on 64×64 InGaAs/InP array detector and Risley-prism scanner

Three-dimensional imaging laser radar with a photon-counting avalanche photodiode array and microchip laser

Asynchronous Single-Photon 3D Imaging

Long-Range Daytime 3D Imaging Lidar With Short Acquisition Time Based on 64×64 Gm-APD Array

Real-time 3D reconstruction from single-photon lidar data using plug-and-play point cloud denoisers

Circular Subwavelength Photodetectors for 3D Space Exploration

An 80 × 25 pixel CMOS single-photon range image sensor with a flexible on-chip time gating topology for solid state 3D scanning

Three-dimensional single-pixel imaging with far fewer measurements than effective image pixels