Abstract:Existing unified methods typically treat multi-degradation image restoration as a multi-task learning problem. Despite performing effectively compared to single degradation restoration methods, they overlook the utilization of commonalities and specificities within multi-task restoration, thereby impeding the model's performance. Inspired by the success of deep generative models and fine-tuning techniques, we proposed a universal image restoration framework based on multiple low-rank adapters (LoRA) from multi-domain transfer learning. Our framework leverages the pre-trained generative model as the shared component for multi-degradation restoration and transfers it to specific degradation image restoration tasks using low-rank adaptation. Additionally, we introduce a LoRA composing strategy based on the degradation similarity, which adaptively combines trained LoRAs and enables our model to be applicable for mixed degradation restoration. Extensive experiments on multiple and mixed degradations demonstrate that the proposed universal image restoration method not only achieves higher fidelity and perceptual image quality but also has better generalization ability than other unified image restoration models. Our code is available at <a class="link-external link-https" href="https://github.com/Justones/UIR-LoRA" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in multi - degraded image restoration tasks, although existing unified methods are superior in performance to single - degraded restoration methods, they fail to fully utilize the commonalities and characteristics in multi - task restoration, thus limiting the performance of the model. Specifically, the paper points out that current methods usually regard multi - degraded image restoration as a multi - task learning problem, but these methods do not allocate specific parameters for different degradations, resulting in gradient conflicts and hindering the further improvement of the unified model's performance. Therefore, the paper proposes a general - purpose image restoration framework (UIR - LoRA) based on Multiple Low - Rank Adaptation (LoRA), aiming to improve the performance and generalization ability of multi - degraded and mixed - degraded image restoration by utilizing the shared components of pre - trained generative models and low - rank adaptation for specific degradations. The main contributions of the paper include: 1. **From the perspective of multi - domain transfer learning**, a general - purpose image restoration framework based on multi - low - rank adaptation is proposed, which uses pre - trained generative models as shared components for multi - degraded restoration tasks and uses different LoRAs for efficient transfer of multiple degradations. 2. **An LoRA combination strategy based on degradation similarity is introduced**, which can adaptively combine trained LoRAs to make the model suitable for the restoration of mixed - degraded images. 3. **Through extensive experimental verification**, it is proved that the proposed general - purpose image restoration method not only achieves higher fidelity and perceptual image quality in various degradations and mixed - degradation restorations, but also has better generalization ability than other unified models. Through these contributions, the paper aims to overcome the limitations of existing methods in handling multi - degraded and mixed - degraded image restoration tasks and provide a more efficient, more flexible and better - performing solution.

UIR-LoRA: Achieving Universal Image Restoration through Multiple Low-Rank Adaptation

LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restoration

Efficient Image Restoration through Low-Rank Adaptation and Stable Diffusion XL

Multi-modal Degradation Feature Learning for Unified Image Restoration Based on Contrastive Learning

Chain-of-Restoration: Multi-Task Image Restoration Models are Zero-Shot Step-by-Step Universal Image Restorers

LLMRA: Multi-modal Large Language Model based Restoration Assistant

Ingredient-Oriented Multi-Degradation Learning for Image Restoration

Reference-based Multi-stage Progressive Restoration for Multi-degraded Images

Contribution-based Low-Rank Adaptation with Pre-training Model for Real Image Restoration

Low-rank Adaptation-based All-Weather Removal for Autonomous Navigation

All-in-one Multi-degradation Image Restoration Network via Hierarchical Degradation Representation

OneRestore: A Universal Restoration Framework for Composite Degradation

Universal Image Restoration with Text Prompt Diffusion

Mixed Degradation Image Restoration via Local Dynamic Optimization and Conditional Embedding

Efficient Degradation-aware Any Image Restoration

Rdm-Ir: Task-Adaptive Deep Unfolding Network for All-in-One Image Restoration

DRM-IR: Task-Adaptive Deep Unfolding Network for All-In-One Image Restoration

LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Models

Enhancing Local–Global Representation Learning for Image Restoration

LIR: A Lightweight Baseline for Image Restoration

In-training Restoration Models Matter: Data Augmentation for Degraded-reference Image Quality Assessment