Abstract:Video snapshot compressive imaging (SCI) uses a low-speed 2D detector to capture high-speed scene, where the dynamic scene is modulated by different masks and then compressed into a snapshot measurement. Following this, a reconstruction algorithm is needed to reconstruct the high-speed video frames. Although state-of-the-art (SOTA) deep learning-based reconstruction algorithms have achieved impressive results, they still face the following challenges due to excessive model complexity and GPU memory limitations: (1) These models need high computational cost, and (2) They are usually unable to reconstruct large-scale video frames at high compression ratios. To address these issues, we develop an efficient network for video SCI by using hierarchical residual-like connections and hybrid CNN-Transformer structure within a single residual block, dubbed EfficientSCI++ . The EfficientSCI++ network can well explore spatial-temporal correlation using convolution in the spatial domain and Transformer in the temporal domain , respectively. We are the first time to demonstrate that a UHD color video ( ) with high compression ratio (40) can be reconstructed from a snapshot 2D measurement using a single end-to-end deep learning model with PSNR above 34 dB. Moreover, a mixed-precision model is trained to further accelerate the video SCI reconstruction process and save memory footprint. Extensive results on both simulation and real data demonstrate that, compared with precious SOTA methods, our proposed EfficientSCI++ and EfficientSCI can achieve comparable reconstruction quality with much cheaper computational cost and better real-time performance. Code is available at https://github.com/mcao92/EfficientSCI-plus-plus.

Key frames assisted hybrid encoding for photorealistic compressive video sensing

Key Frames Assisted Hybrid Encoding for High-Quality Compressive Video Sensing

Adaptive temporal compressive sensing for video with motion estimation

Deep Optics for Video Snapshot Compressive Imaging

Hybrid CNN-Transformer Architecture for Efficient Large-Scale Video Snapshot Compressive Imaging

Event-Enhanced Snapshot Compressive Videography at 10K FPS

Deep Motion Regularizer for Video Snapshot Compressive Imaging

EfficientSCI: Densely Connected Network with Space-time Factorization for Large-scale Video Snapshot Compressive Imaging

Deep learning for video compressive sensing

MetaSCI: Scalable and Adaptive Reconstruction for Video Compressive Sensing

Motion-aware Dynamic Graph Neural Network for Video Compressive Sensing

Video Compressed Sensing Using a Convolutional Neural Network

Compressive Sensing Reconstruction for Video: an Adaptive Approach Based on Motion Estimation.

Distributed Compressed Video Sensing Based on Key Frame Secondary Reconstruction

High-Performance Distributed Compressive Video Sensing: Jointly Exploiting the HEVC Motion Estimation and the ℓ1 – ℓ1 Reconstruction

Video Compressive Sensing for Spatial Multiplexing Cameras using Motion-Flow Models

Memory-Efficient Network for Large-scale Video Compressive Sensing

Video Compressive Sensing Reconstruction Via Reweighted Residual Sparsity

A Simple Low-bit Quantization Framework for Video Snapshot Compressive Imaging

Distributed Compressed Video Sensing Based on the Optimization of Hypothesis Set Update Technique

A New Multiple Frames Decoding and Frame Wise Measurement for Compressed Video Sensing