Abstract:Abstract Recently, convolutional neural networks (CNNs) and vision transformers (ViTs) have emerged as powerful tools for image restoration (IR). Nonetheless, they encountered some limitations due to their characteristics, such as CNNs sacrificing global reception and ViTs requiring large memory and graphics resources. To address these limitations and explore an alternative approach for improved IR performance, we propose two clustering-based frameworks for general IR tasks, which are style-guided context cluster U-Net (SCoC-UNet) and style-guided clustered point interaction U-Net (SCPI-UNet). The SCoC-UNet adopts a U-shaped architecture, comprising position embedding, Encoder, Decoder, and reconstruction block. Specifically, the input low-quality image is viewed as a set of unorganized points, each of which is first given location information by the continuous relative position embedding method. These points are then fed into a symmetric Encoder and Decoder which utilize style-guided context cluster (SCoC) blocks to extract potential context features and high-frequency information. Although SCoC-UNet has obtained decent performance for image restoration, its SCoC block can only capture connectivity at points within the same cluster, which may ignore long-range dependencies in different clusters. To address this issue, we further propose a SCPI-UNet based on SCoC-UNet, which leverages a style-guided clustered point interaction (SCPI) block in place of the SCoC block. The SCPI block utilizes a cross-attention mechanism to establish the connections of feature points between different clusters. Extensive experimental results demonstrate that the proposed SCoC-UNet and SCPI-UNet can handle several typical IR tasks (i.e., JPEG compression artifact reduction, image denoising, and super-resolution) and achieve superior quantitative and qualitative performance over some state-of-the-art methods.

Image Diversity Restoration Algorithm Based on Unsupervised Sample Guidance

Image restoration for real-world under-display imaging

A Unified Conditional Framework for Diffusion-based Image Restoration

A Double Feature Fusion Network with Progressive Learning for Sharper Inpainting

Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration

Deep Generalized Unfolding Networks for Image Restoration

Universal Image Restoration with Text Prompt Diffusion

UDC-UNet: Under-Display Camera Image Restoration via U-Shape Dynamic Network

Selective Hourglass Mapping for Universal Image Restoration Based on Diffusion Model

Mixed Degradation Image Restoration via Local Dynamic Optimization and Conditional Embedding

High-dimensional Assisted Generative Model for Color Image Restoration

Variational Image Restoration Network.

Decoupled Data Consistency with Diffusion Purification for Image Restoration

Efficient image restoration with style-guided context cluster and interaction

HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models

Variational Image Restoration Network

Supervise-Assisted Self-Supervised Deep-Learning Method for Hyperspectral Image Restoration

Generative Adversarial Unsupervised Image Restoration in Hybrid Degradation Scenes

From Posterior Sampling to Meaningful Diversity in Image Restoration

Multiscale Synergism Ensemble Progressive and Contrastive Investigation for Image Restoration