Abstract:Single-photon avalanche diodes (SPADs) are advanced sensors capable of detecting individual photons and recording their arrival times with picosecond resolution using time-correlated Single-Photon Counting detection techniques. They are used in various applications, such as LiDAR, and can capture high-speed sequences of binary single-photon images, offering great potential for reconstructing 3D environments with high motion dynamics. To complement single-photon data, they are often paired with conventional passive cameras, which capture high-resolution (HR) intensity images at a lower frame rate. However, 3D reconstruction from SPAD data faces challenges. Aggregating multiple binary measurements improves precision and reduces noise but can cause motion blur in dynamic scenes. Additionally, SPAD arrays often have lower resolution than passive cameras. To address these issues, we propose a novel computational imaging algorithm to improve the 3D reconstruction of moving scenes from SPAD data by addressing the motion blur and increasing the native spatial resolution. We adopt a plug-and-play approach within an optimization scheme alternating between guided video super-resolution of the 3D scene, and precise image realignment using optical flow. Experiments on synthetic data show significantly improved image resolutions across various signal-to-noise ratios and photon levels. We validate our method using real-world SPAD measurements on three practical situations with dynamic objects. First on fast-moving scenes in laboratory conditions at short range; second very low resolution imaging of people with a consumer-grade SPAD sensor from STMicroelectronics; and finally, HR imaging of people walking outdoors in daylight at a range of 325 meters under eye-safe illumination conditions using a short-wave infrared SPAD camera. These results demonstrate the robustness and versatility of our approach.

High-resolution multi-spectral snapshot 3D imaging with a SPAD array camera

Scannerless Non-Line-of-sight Three Dimensional Imaging with a 32x32 SPAD Array

Optimization of System Design and Calibration Algorithm for SPAD-Based LiDAR Imager

Single photon imaging based on a photon driven sparse sampling

PCA-based real-time single-photon 3D imaging method

Solid-state Multi-beam Scanning Single-photon Lidar System Based on Cascaded VIPA and SPAD Array

Airborne Single-Photon LiDAR Towards a Small-Sized and Low-Power Payload

FoveaSPAD: Exploiting Depth Priors for Adaptive and Efficient Single-Photon 3D Imaging

Single-photon depth imaging using a photodetector with a wire-bonding quenching resistor

Simulating single-photon detector array sensors for depth imaging

SPADs and SiPMs Arrays for Long-Range High-Speed Light Detection and Ranging (LiDAR)

Deep Learning Based Single-Photon 3D Imaging with Multiple Returns

A Plug-and-Play Algorithm for 3D Video Super-Resolution of Single-Photon LiDAR data

A 32×32 Flash LiDAR SPAD Sensor with Up-to-1kfps Motional Target Detection by Threshold-adaptive 2D Dynamic Vision

Asynchronous Single-Photon 3D Imaging

A 256 × 256 LiDAR Imaging System Based on a 200 mW SPAD-Based SoC with Microlens Array and Lightweight RGB-Guided Depth Completion Neural Network

Scanning single-pixel imaging lidar

High-resolution 3D Imaging Through Dense Camouflage Nets Using Single-Photon LiDAR

Risley-prism-based multi-beam scanning LiDAR for high-resolution three-dimensional imaging

Subpixel Three-Dimensional Laser Imaging with a Downscaled Avalanche Photodiode Array Using Code Division Multiple Access

Subpixel Three Dimensional Imaging Using Two Dimensional Optical Encoding with a Downscaled Avalanche Photodiode Array