Abstract:Video snapshot compressive imaging (SCI) uses a low-speed 2D detector to capture high-speed scene, where the dynamic scene is modulated by different masks and then compressed into a snapshot measurement. Following this, a reconstruction algorithm is needed to reconstruct the high-speed video frames. Although state-of-the-art (SOTA) deep learning-based reconstruction algorithms have achieved impressive results, they still face the following challenges due to excessive model complexity and GPU memory limitations: (1) These models need high computational cost, and (2) They are usually unable to reconstruct large-scale video frames at high compression ratios. To address these issues, we develop an efficient network for video SCI by using hierarchical residual-like connections and hybrid CNN-Transformer structure within a single residual block, dubbed EfficientSCI++ . The EfficientSCI++ network can well explore spatial-temporal correlation using convolution in the spatial domain and Transformer in the temporal domain , respectively. We are the first time to demonstrate that a UHD color video ( ) with high compression ratio (40) can be reconstructed from a snapshot 2D measurement using a single end-to-end deep learning model with PSNR above 34 dB. Moreover, a mixed-precision model is trained to further accelerate the video SCI reconstruction process and save memory footprint. Extensive results on both simulation and real data demonstrate that, compared with precious SOTA methods, our proposed EfficientSCI++ and EfficientSCI can achieve comparable reconstruction quality with much cheaper computational cost and better real-time performance. Code is available at https://github.com/mcao92/EfficientSCI-plus-plus.

Transformer-based flexible sampling ratio compressed ghost imaging

Compressive Sensing Ghost Imaging Based On Neighbor Similarity

Compressive sensing ghost imaging based on image gradient

Foveated Ghost Imaging Based on Deep Learning

Compressed Sensing Fourier Single Pixel Imaging Algorithm Based on Joint Discrete Gradient and Non-Local Self-Similarity Priors

Computational ghost imaging with deep compressed sensing

Compressive Sensing Ghost Imaging Object Detection Using Generative Adversarial Networks

Computational ghost imaging using the dilated ghost network

Fast high quality computational ghost imaging based on saliency variable sampling detection

Real-time physical compression computational ghost imaging based on array spatial light field modulation and deep learning

Single Photon Counting Compressive Imaging Using a Generative Model Optimized Via Sampling and Transfer Learning.

Color computational ghost imaging by deep learning based on simulation data training

High-speed computational ghost imaging with compressed sensing based on a convolutional neural network

Deep-learning-based Single-Photon-counting Compressive Imaging Via Jointly Trained Subpixel Convolution Sampling.

Local-enhanced transformer for single-pixel imaging

Ghost Imaging Based on Recurrent Neural Network

Semantic ghost imaging based on semantic coding

Compressive Ghost Imaging in Scattering Media Guided by Region of Interest

Compressed ghost edge imaging

Super-resolution ghost imaging via compressive sampling reconstruction

Hybrid CNN-Transformer Architecture for Efficient Large-Scale Video Snapshot Compressive Imaging