Abstract:Image restoration models often face the simultaneous interaction of multiple degradations in real-world scenarios. Existing approaches typically handle single or composite degradations based on scene descriptors derived from text or image embeddings. However, due to the varying proportions of different degradations within an image, these scene descriptors may not accurately differentiate between degradations, leading to suboptimal restoration in practical applications. To address this issue, we propose a novel Transformer-based restoration framework, AllRestorer. In AllRestorer, we enable the model to adaptively consider all image impairments, thereby avoiding errors from scene descriptor misdirection. Specifically, we introduce an All-in-One Transformer Block (AiOTB), which adaptively removes all degradations present in a given image by modeling the relationships between all degradations and the image embedding in latent space. To accurately address different variations potentially present within the same type of degradation and minimize ambiguity, AiOTB utilizes a composite scene descriptor consisting of both image and text embeddings to define the degradation. Furthermore, AiOTB includes an adaptive weight for each degradation, allowing for precise control of the restoration intensity. By leveraging AiOTB, AllRestorer avoids misdirection caused by inaccurate scene descriptors, achieving a 5.00 dB increase in PSNR compared to the baseline on the CDD-11 dataset.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the problem of composite degradations encountered in the image inpainting task. Specifically, existing methods, when dealing with single or composite degradations, usually rely on scene descriptors derived from text or image embeddings. However, due to the different proportions of different degradations in the image, these scene descriptors may not accurately distinguish different degradation types, resulting in unsatisfactory inpainting effects in practical applications. To meet this challenge, the authors propose a new Transformer - based inpainting framework - **AllRestorer**. AllRestorer can adaptively consider all image damages by introducing a new module named **All - in - One Transformer Block (AiOTB)**, avoiding errors caused by misleading scene descriptors. Specifically: 1. **Composite scene descriptors**: The composite scene descriptors that combine image and text embeddings ensure a more accurate representation of degradation types. 2. **Adaptive weights**: Adaptive weights are assigned to each degradation type to precisely control the inpainting intensity and adapt to the proportional changes of various degradations in different scenarios. 3. **All - in - One Attention (AiOA) mechanism**: By modeling the relationship between degradations and image embeddings, all degradations are removed in the latent space. The experimental results of AllRestorer on the CDD - 11 dataset show that it performs better than existing methods in dealing with composite degradations, with a 5.00 dB improvement in PSNR. In addition, real - world tests also verify the potential of AllRestorer in practical applications. ### Summary The main contributions of this paper include: - Proposing a new inpainting scheme, AllRestorer, which can effectively handle multiple types of composite degradations. - Introducing composite scene descriptors based on text and image embeddings to ensure an accurate representation of degradation types. - Designing an adaptive weight mechanism to adapt to the changes in the proportion of degradations in different scenarios and ensure the precise control of inpainting intensity. Through these innovations, AllRestorer has demonstrated state - of - the - art performance on both synthetic and real datasets.

AllRestorer: All-in-One Transformer for Image Restoration under Composite Degradations

Restorer: Removing Multi-Degradation with All-Axis Attention and Prompt Guidance

OneRestore: A Universal Restoration Framework for Composite Degradation

Learning Frequency-Aware Dynamic Transformers for All-In-One Image Restoration

Restormer: Efficient Transformer for High-Resolution Image Restoration

All-in-one Multi-degradation Image Restoration Network via Hierarchical Degradation Representation

Harmony in Diversity: Improving All-in-One Image Restoration Via Multi-Task Collaboration

Adapt or Perish: Adaptive Sparse Transformer with Attentive Feature Refinement for Image Restoration

Multi-Weather Degradation-Aware Transformer for Image Restoration

Dual-former: Hybrid Self-attention Transformer for Efficient Image Restoration

Progressive Convolutional Transformer for Image Restoration

Rdm-Ir: Task-Adaptive Deep Unfolding Network for All-in-One Image Restoration

Mixed Degradation Image Restoration via Local Dynamic Optimization and Conditional Embedding

DRM-IR: Task-Adaptive Deep Unfolding Network for All-In-One Image Restoration

Accurate Image Restoration with Attention Retractable Transformer

Joint multi-dimensional dynamic attention and transformer for general image restoration

Multi-modal Degradation Feature Learning for Unified Image Restoration Based on Contrastive Learning

Referring Flexible Image Restoration

Reference-based Multi-stage Progressive Restoration for Multi-degraded Images

Revitalizing Convolutional Network for Image Restoration

Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration