Abstract:Single Photon Avalanche Photodiodes (SPADs) have recently received a lot of attention in imaging and vision applications due to their excellent performance in low-light conditions, as well as their ultra-high temporal resolution. Unfortunately, like many evolving sensor technologies, image sensors built around SPAD technology currently suffer from a low pixel count. In this work, we investigate a simple, low-cost, and compact optical coding camera design that supports high-resolution image reconstructions from raw measurements with low pixel counts. We demonstrate this approach for regular intensity imaging, depth imaging, as well transient imaging. Our method uses an end-to-end framework to simultaneously optimize the optical design and a reconstruction network for obtaining super-resolved images from raw measurements. The optical design space is that of an engineered point spread function (implemented with diffractive optics), which can be considered an optimized anti-aliasing filter to preserve as much high-resolution information as possible despite imaging with a low pixel count, low fill-factor SPAD array. We further investigate a deep network for reconstruction. The effectiveness of this joint design and reconstruction approach is demonstrated for a range of different applications, including high-speed imaging, and time of flight depth imaging, as well as transient imaging. While our work specifically focuses on low-resolution SPAD sensors, similar approaches should prove effective for other emerging image sensor technologies with low pixel counts and low fill-factors.

End-to-end Snapshot Compressed Super-Resolution Imaging with Deep Optics

Fast Image Super-Resolution for a Dual-Resolution Camera

Deeply Learned Broadband Encoding Stochastic Hyperspectral Imaging.

Single-shot Real-Time Compressed Ultrahigh-Speed Imaging Enabled by a Snapshot-to-video Autoencoder

10-mega pixel snapshot compressive imaging with a hybrid coded aperture

Ten-mega-pixel snapshot compressive imaging with a hybrid coded aperture

Snapshot-to-video autoencoder for compressed ultrahigh-speed imaging

Parallel compressive super-resolution imaging with wide field-of-view based on physics enhanced network

Deep Optics for Video Snapshot Compressive Imaging

Single-Exposure Frequency-Domain Amplitude Encoding Compressive Imaging

End-to-end Learned, Optically Coded Super-resolution SPAD Camera

Compressed Ultrafast Photography by Multi-Encoding Imaging

Superresolution imaging by dynamic single-pixel compressive sensing system

Sinusoidal Sampling Enhanced Compressive Camera for High Speed Imaging

Super-Resolution Imaging Through Scattering Medium Based On Parallel Compressed Sensing

Super Resolution Optic Three-dimensional Imaging Based on Compressed Sensing

Deep-learning-based Single-Photon-counting Compressive Imaging Via Jointly Trained Subpixel Convolution Sampling.

Deep learning for video compressive sensing

Snapshot Multiplexed Imaging Based on Compressive Sensing.

Event-Enhanced Snapshot Compressive Videography at 10K FPS

Snapshot Interferometric 3D Imaging by Compressive Sensing and Deep Learning