Abstract:In recent years, deep unfolding networks (DUNs) have received widespread attention in the field of compressed sensing (CS) reconstruction due to their good interpretability and strong mapping capabilities. However, existing DUNs often improve the reconstruction effect at the expense of a large number of parameters, and there is the problem of information loss in long-distance feature transmission. Based on the above problems, we propose an unfolded network architecture that mixes Transformer and large kernel convolution to achieve sparse sampling and reconstruction of natural images, namely, a reconstruction network based on Transformer and convolution (TCR-Net). The Transformer framework has the inherent ability to capture global context through a self-attention mechanism, which can effectively solve the challenge of long-range dependence on features. TCR-Net is an end-to-end two-stage architecture. First, a data-driven pre-trained encoder is used to complete the sparse representation and basic feature extraction of image information. Second, a new attention mechanism is introduced to replace the self-attention mechanism in Transformer, and a hybrid Transformer and convolution module based on optimization-inspired is designed. Its iterative process leads to the unfolding framework, which approximates the original image stage by stage. Experimental results show that TCR-Net outperforms existing state-of-the-art CS methods while maintaining fast computational speed. Specifically, when the CS ratio is 0.10, the average PSNR on the test set used in this paper is improved by at least 0.8%, the average SSIM is improved by at least 1.5%, and the processing speed is higher than 70FPS. These quantitative results show that our method has high computational efficiency while ensuring high-quality image restoration.

Gates-Controlled Deep Unfolding Network for Image Compressed Sensing

Rethinking Performance Gains in Image Dehazing Networks

Deep Unfolding Network for Image Compressed Sensing by Content-adaptive Gradient Updating and Deformation-invariant Non-local Modeling

Memory-Augmented Deep Unfolding Network for Compressive Sensing

Dynamic Path-Controllable Deep Unfolding Network for Compressive Sensing

High-Throughput Deep Unfolding Network for Compressive Sensing MRI

Optimization-inspired Cumulative Transmission Network for image compressive sensing

Fast Hierarchical Deep Unfolding Network for Image Compressed Sensing

ASAUN-CS: Adaptive Stage Activated Unfolding Network for Compressive Sensing

MCU-Net: A Multi-prior Collaborative Deep Unfolding Network with Gates-controlled Spatial Attention for Accelerated MR Image Reconstruction

D3C2-Net: Dual-Domain Deep Convolutional Coding Network for Compressive Sensing

LR-CSNet: Low-Rank Deep Unfolding Network for Image Compressive Sensing

SODAS-Net: Side-Information-Aided Deep Adaptive Shrinkage Network for Compressive Sensing

SALSA-Net: Explainable Deep Unrolling Networks for Compressed Sensing

Multi-Channel Representation Learning Enhanced Unfolding Multi-Scale Compressed Sensing Network for High Quality Image Reconstruction

Gated Recurrent Fusion UNet for Depth Completion

Hybrid Transformer and Convolution for Image Compressed Sensing

A Dictionary Learning Based Unsupervised Neural Network for Single Image Compressed Sensing

Degradation Estimation Recurrent Neural Network with Local and Non-Local Priors for Compressive Spectral Imaging

Optimization-Inspired Cross-Attention Transformer for Compressive Sensing

Deep Memory-Augmented Proximal Unrolling Network for Compressive Sensing