Abstract:Video snapshot compressive imaging (SCI) uses a low-speed 2D detector to capture high-speed scene, where the dynamic scene is modulated by different masks and then compressed into a snapshot measurement. Following this, a reconstruction algorithm is needed to reconstruct the high-speed video frames. Although state-of-the-art (SOTA) deep learning-based reconstruction algorithms have achieved impressive results, they still face the following challenges due to excessive model complexity and GPU memory limitations: (1) These models need high computational cost, and (2) They are usually unable to reconstruct large-scale video frames at high compression ratios. To address these issues, we develop an efficient network for video SCI by using hierarchical residual-like connections and hybrid CNN-Transformer structure within a single residual block, dubbed EfficientSCI++ . The EfficientSCI++ network can well explore spatial-temporal correlation using convolution in the spatial domain and Transformer in the temporal domain , respectively. We are the first time to demonstrate that a UHD color video ( ) with high compression ratio (40) can be reconstructed from a snapshot 2D measurement using a single end-to-end deep learning model with PSNR above 34 dB. Moreover, a mixed-precision model is trained to further accelerate the video SCI reconstruction process and save memory footprint. Extensive results on both simulation and real data demonstrate that, compared with precious SOTA methods, our proposed EfficientSCI++ and EfficientSCI can achieve comparable reconstruction quality with much cheaper computational cost and better real-time performance. Code is available at https://github.com/mcao92/EfficientSCI-plus-plus.

An Efficient Transformer For Demosaicing Via Compressed Multi-Branch Attention Mechanism.

Learning to Joint Remosaic and Denoise in Quad Bayer CFA via Universal Multi-scale Channel Attention Network.

MSFA-Frequency-Aware Transformer for Hyperspectral Images Demosaicing

PPI Edge Infused Spatial-Spectral Adaptive Residual Network for Multispectral Filter Array Image Demosaicing

MCFD: A Hardware-Efficient Noniterative Multicue Fusion Demosaicing Algorithm.

A Snapshot Multi-Spectral Demosaicing Method for Multi-Spectral Filter Array Images Based on Channel Attention Network

Joint learning of RGBW color filter arrays and demosaicking

A Patch Aware Multiple Dictionary Framework For Demosaicing

Universal Demosaicking of Color Filter Arrays

Efficient Unified Demosaicing for Bayer and Non-Bayer Patterned Image Sensors

Optimized Color Filter Arrays for Sparse Representation-Based Demosaicking

Efficient Training Procedures for Multi-Spectra Demosaicing

Efficient Depth Fusion Transformer for Aerial Image Semantic Segmentation

An Efficient Adaptive Interpolation for Bayer CFA Demosaicking

Hybrid CNN-Transformer Architecture for Efficient Large-Scale Video Snapshot Compressive Imaging

Effective Color Filter Array Demosaicing Method

DemosaicFormer: Coarse-to-Fine Demosaicing Network for HybridEVS Camera

Efficient Mixed Transformer for Single Image Super-Resolution

Efficient Concertormer for Image Deblurring and Beyond

Color demosaicking via fully directional estimation