Abstract:Exploiting pre-trained diffusion models for restoration has recently become a favored alternative to the traditional task-specific training approach. Previous works have achieved noteworthy success by limiting the solution space using explicit degradation models. However, these methods often fall short when faced with complex degradations as they generally cannot be precisely modeled. In this paper, we propose PGDiff by introducing partial guidance, a fresh perspective that is more adaptable to real-world degradations compared to existing works. Rather than specifically defining the degradation process, our approach models the desired properties, such as image structure and color statistics of high-quality images, and applies this guidance during the reverse diffusion process. These properties are readily available and make no assumptions about the degradation process. When combined with a diffusion prior, this partial guidance can deliver appealing results across a range of restoration tasks. Additionally, PGDiff can be extended to handle composite tasks by consolidating multiple high-quality image properties, achieved by integrating the guidance from respective tasks. Experimental results demonstrate that our method not only outperforms existing diffusion-prior-based approaches but also competes favorably with task-specific models.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to use pre - trained diffusion models to achieve high - quality image restoration in image restoration tasks without relying on assumptions about specific degradation processes. Existing methods usually require explicit modeling of the degradation process, which limits their generalization ability when facing complex or unknown degradation situations. This paper proposes a new method - PGDiff (Partial Guidance for Diffusion Models). By introducing the concept of partial guidance, it directly models the desired properties of high - quality images instead of the degradation process. This method can more flexibly deal with complex degradations in the real world and is applicable to a variety of image restoration tasks, such as blind face restoration, face coloring, face inpainting and old photo restoration. Specifically, the main contributions of PGDiff include: 1. **New Adaptation Strategy**: A new concept that does not require pre - assumptions about the degradation process is proposed, that is, to adapt the diffusion model for restoration by guiding the easily accessible properties in the high - quality image space (such as image structure and color statistics) during the denoising process. 2. **Partial Guidance**: A simple and effective method - partial guidance is designed. It is applicable to a wide range of image restoration and enhancement tasks and allows for the flexible combination of different guidance methods for complex tasks. 3. **Experimental Verification**: Through extensive experiments, the effectiveness of PGDiff on a variety of challenging tasks (including blind face restoration and old photo restoration) is proven, showing its performance superior to the existing state - of - the - art methods. This method not only improves the quality of image restoration but also enhances the generalization ability and adaptability of the model, especially when dealing with complex and diverse degradation types.

PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance

PDGrad: Guiding Diffusion Model for Reference-Based Blind Face Restoration with Pivot Direction Gradient Guidance

Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration

Gradient Guidance for Diffusion Models: An Optimization Perspective

Perceptual Similarity guidance and text guidance optimization for Editing Real Images using Guided Diffusion Models

DiffLoss: unleashing diffusion model as constraint for training image restoration network

DifFace: Blind Face Restoration with Diffused Error Contraction

Decoupled Data Consistency with Diffusion Purification for Image Restoration

Training-free Prior Guided Diffusion Model for Zero-Reference Low-Light Image Enhancement

AGLLDiff: Guiding Diffusion Models Towards Unsupervised Training-free Real-world Low-light Image Enhancement

PFDiff: Training-free Acceleration of Diffusion Models through the Gradient Guidance of Past and Future

Self-Guided Diffusion Models

Towards Unsupervised Blind Face Restoration using Diffusion Prior

Frequency-Aware Guidance for Blind Image Restoration via Diffusion Models

ReviveDiff: A Universal Diffusion Model for Restoring Images in Adverse Weather Conditions

Collaborative Diffusion for Multi-Modal Face Generation and Editing

Generative Diffusion Prior for Unified Image Restoration and Enhancement

Guided Diffusion from Self-Supervised Diffusion Features

Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model

Transfer Learning for Diffusion Models