Abstract:Single-photon avalanche diodes (SPADs) are advanced sensors capable of detecting individual photons and recording their arrival times with picosecond resolution using time-correlated Single-Photon Counting detection techniques. They are used in various applications, such as LiDAR, and can capture high-speed sequences of binary single-photon images, offering great potential for reconstructing 3D environments with high motion dynamics. To complement single-photon data, they are often paired with conventional passive cameras, which capture high-resolution (HR) intensity images at a lower frame rate. However, 3D reconstruction from SPAD data faces challenges. Aggregating multiple binary measurements improves precision and reduces noise but can cause motion blur in dynamic scenes. Additionally, SPAD arrays often have lower resolution than passive cameras. To address these issues, we propose a novel computational imaging algorithm to improve the 3D reconstruction of moving scenes from SPAD data by addressing the motion blur and increasing the native spatial resolution. We adopt a plug-and-play approach within an optimization scheme alternating between guided video super-resolution of the 3D scene, and precise image realignment using optical flow. Experiments on synthetic data show significantly improved image resolutions across various signal-to-noise ratios and photon levels. We validate our method using real-world SPAD measurements on three practical situations with dynamic objects. First on fast-moving scenes in laboratory conditions at short range; second very low resolution imaging of people with a consumer-grade SPAD sensor from STMicroelectronics; and finally, HR imaging of people walking outdoors in daylight at a range of 325 meters under eye-safe illumination conditions using a short-wave infrared SPAD camera. These results demonstrate the robustness and versatility of our approach.

Non-Line-of-Sight Three-Dimensional Imaging with a Single-Pixel Camera

3D RGB Non-Line-Of-Sight single-pixel imaging

Three Dimensional Reconstruction Using a Lenslet Light Field Camera

Non-line-of-sight imaging using a time-gated single photon avalanche diode

Two-edge-resolved three-dimensional non-line-of-sight imaging with an ordinary camera

Subpixel Three Dimensional Imaging Using Two Dimensional Optical Encoding with a Downscaled Avalanche Photodiode Array

Single-Photon 3D Imaging with Equi-Depth Photon Histograms

Subpixel Three-Dimensional Laser Imaging with a Downscaled Avalanche Photodiode Array Using Code Division Multiple Access

Single-photon 3D imaging with deep sensor fusion

Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second

Single photon imaging based on a photon driven sparse sampling

Real-time 3D reconstruction from single-photon lidar data using plug-and-play point cloud denoisers

High-resolution multi-spectral snapshot 3D imaging with a SPAD array camera

PCA-based real-time single-photon 3D imaging method

Deep Learning Based Single-Photon 3D Imaging with Multiple Returns

Time-encoded single-pixel 3D imaging

Asynchronous Single-Photon 3D Imaging

A Plug-and-Play Algorithm for 3D Video Super-Resolution of Single-Photon LiDAR data

Three-dimensional single-pixel imaging with far fewer measurements than effective image pixels

Single-pixel 3D imaging based on fusion temporal data of single-photon detector and millimeter-wave radar

Megapixel time-gated SPAD image sensor for 2D and 3D imaging applications