SASA: Saliency-Aware Self-Adaptive Snapshot Compressive Imaging

Yaping Zhao,Edmund Y. Lam

2023-12-30

Abstract:The ability of snapshot compressive imaging (SCI) systems to efficiently capture high-dimensional (HD) data depends on the advent of novel optical designs to sample the HD data as two-dimensional (2D) compressed measurements. Nonetheless, the traditional SCI scheme is fundamentally limited, due to the complete disregard for high-level information in the sampling process. To tackle this issue, in this paper, we pave the first mile toward the advanced design of adaptive coding masks for SCI. Specifically, we propose an efficient and effective algorithm to generate coding masks with the assistance of saliency detection, in a low-cost and low-power fashion. Experiments demonstrate the effectiveness and efficiency of our approach. Code is available at:

Image and Video Processing

What problem does this paper attempt to address?

The paper aims to address the issue of traditional Snapshot Compressive Imaging (SCI) systems neglecting high-level information during the sampling process. Specifically, traditional SCI methods use random binary masks during sampling, treating all pixels equally without considering important information such as objects or salience in the image. This approach leads to inefficient resource allocation, wasting computation on irrelevant content, and operates independently of subsequent computer vision tasks, limiting sampling and design efficiency. To solve these problems, the authors propose a method called Salience-Aware Self-Adaptive Snapshot Compressive Imaging (SASA), which introduces salience detection to generate adaptive coding masks. This method can dynamically adjust the sampling probability based on the importance of image regions under low power and low cost, thereby improving the quality of compressed measurements and the performance of reconstruction results. Experimental results show that compared to traditional SCI methods, SASA can significantly enhance image reconstruction effects, with improvements of 0.2 to 0.5dB in PSNR and 0.01 to 0.03 in SSIM. Additionally, SASA has real-time processing capabilities, with an average processing speed of up to 250 frames per second, suitable for a single laptop CPU.

SASA: Saliency-Aware Self-Adaptive Snapshot Compressive Imaging

Compressive Single-Pixel Spectral Imaging with Spatial-Spectral Modulation Optimization Via Coherence Minimization

Hybrid Compressive Spectral Imaging with Coded Aperture Optimization Based on Coherence Minimization

Snapshot Compressive Imaging: Principle, Implementation, Theory, Algorithms and Applications

Spatial-spectral Encoding and Dictionary Optimization in Compressive Single-Pixel Hyperspectral Imaging Based on Mutual Coherence Minimization

10-mega pixel snapshot compressive imaging with a hybrid coded aperture

Shearlet Enhanced Snapshot Compressive Imaging

Plug-and-Play Algorithms for Large-scale Snapshot Compressive Imaging

Compact Self-adaptive Coding for Spectral Compressive Sensing

Ten-mega-pixel snapshot compressive imaging with a hybrid coded aperture

D-HAN: an End-to-End Adaptive Neural Network for Snapshot Compressive Imaging

EfficientSCI: Densely Connected Network with Space-time Factorization for Large-scale Video Snapshot Compressive Imaging

Deep-Learning Supervised Snapshot Compressive Imaging Enabled by an End-to-End Adaptive Neural Network.

Deep Optics for Video Snapshot Compressive Imaging

Spatio-Temporal Coherent Snapshot Compressed Imaging with Time-Varying Composite Phase Masks

Plug-and-Play Algorithms for Video Snapshot Compressive Imaging

Rank Minimization for Snapshot Compressive Imaging

Efficient One-Step Diffusion Refinement for Snapshot Compressive Imaging

Adaptive Sampling for Image Compressed Sensing Based on Deep Learning

Dataset for 10-Mega Pixel Snapshot Compressive Imaging with a Hybrid Coded Aperture

SAH-SCI: Self-Supervised Adapter for Efficient Hyperspectral Snapshot Compressive Imaging