Abstract:Recently, various methods have been proposed to solve Image Restoration (IR) tasks using a pre-trained diffusion model leading to state-of-the-art performance. However, most of these methods assume that the degradation operator in the IR task is completely known. Furthermore, a common characteristic among these approaches is that they alter the diffusion sampling process in order to satisfy the consistency with the degraded input image. This choice has recently been shown to be sub-optimal and to cause the restored image to deviate from the data manifold. To address these issues, we propose Blind Image Restoration via fast Diffusion inversion (BIRD) a blind IR method that jointly optimizes for the degradation model parameters and the restored image. To ensure that the restored images lie onto the data manifold, we propose a novel sampling technique on a pre-trained diffusion model. A key idea in our method is not to modify the reverse sampling, i.e., not to alter all the intermediate latents, once an initial noise is sampled. This is ultimately equivalent to casting the IR task as an optimization problem in the space of the input noise. Moreover, to mitigate the computational cost associated with inverting a fully unrolled diffusion model, we leverage the inherent capability of these models to skip ahead in the forward diffusion process using large time steps. We experimentally validate BIRD on several image restoration tasks and show that it achieves state of the art performance on all of them. Our code is available at <a class="link-external link-https" href="https://github.com/hamadichihaoui/BIRD" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper aims to address the problem of blind image restoration (IR). Specifically, most existing methods assume that the degradation operator (i.e., factors causing image quality degradation, such as blur, noise, etc.) is known when handling image restoration tasks. However, in practical applications, this assumption often does not hold because the degradation operator is usually unknown or partially unknown. ### Main Contributions 1. **Proposed a new blind image restoration method (BIRD)**: - BIRD addresses the blind image restoration problem by jointly optimizing the degradation model parameters and the restored image. - This method utilizes a pre-trained diffusion model (DDIM) to transform the inverse restoration problem into a latent variable estimation problem, ensuring that the restored image always lies on the data manifold. 2. **High computational efficiency**: - BIRD improves computational efficiency by avoiding fine-tuning or retraining the diffusion model through fast diffusion model inversion. - By leveraging the inherent capabilities of the diffusion model, it can skip numerous time steps in the forward diffusion process, further reducing computational costs. 3. **Wide applicability**: - BIRD is applicable to various image restoration tasks, including Gaussian deblurring, motion deblurring, super-resolution, and JPEG artifact removal. - This method does not require additional training for each new degradation operator and can be directly applied to new images and new degradation models. ### Method Overview - **Problem Definition**: - The image restoration task can be defined as a maximum a posteriori (MAP) optimization problem, where the goal is to maximize the posterior probability of the restored image given the degraded image. - The degradation model \( H_\eta \) maps the original image \( x \) to the degraded image \( y \), i.e., \( y = H_\eta(x) + n \), where \( n \) is noise. - **Optimization Process**: - BIRD starts from initial noise \( x_T \sim \mathcal{N}(0, I) \) and generates latent variables \( x_0 \) through the pre-trained DDIM model. - By alternately optimizing the restored image \( x_0 \) and the degradation model parameters \( \eta \), it minimizes the restoration loss \( L_{IR} \). - In each iteration, \( x_T \) and \( \eta \) are updated, and a normalization step ensures that the Euclidean norm of \( x_T \) is \( \sqrt{N_x \times N_y} \). ### Experimental Results - **Quantitative Evaluation**: - BIRD achieves state-of-the-art performance on multiple image restoration tasks on the CelebA and ImageNet validation datasets, particularly excelling in PSNR and LPIPS metrics. - **Qualitative Evaluation**: - Compared to existing methods (such as GDP, BlindDPS), BIRD restores images that are more realistic, consistent, and accurate. - **Computational Efficiency**: - BIRD outperforms other methods in terms of runtime and memory usage, especially by adjusting the step size \( \delta_t \) to balance speed and accuracy. ### Conclusion BIRD provides a robust, accurate, and efficient framework for blind image restoration by leveraging pre-trained diffusion models as learning priors. This method is not only suitable for various image restoration tasks but also highly computationally efficient, offering broad practical value.

Blind Image Restoration via Fast Diffusion Inversion

DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior

Efficient Diffusion Model for Image Restoration by Residual Shifting

Taming Generative Diffusion Prior for Universal Blind Image Restoration

DiffIR: Efficient Diffusion Model for Image Restoration

Towards Unsupervised Blind Face Restoration using Diffusion Prior

Denoising Diffusion Models for Plug-and-Play Image Restoration

Blind Inversion using Latent Diffusion Priors

Fast Diffusion EM: a diffusion model for blind inverse problems with application to deconvolution

HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models

Combined Approach of Blind Image Restoration

Photo-Realistic Image Restoration in the Wild with Controlled Vision-Language Models

InstantIR: Blind Image Restoration with Instant Generative Reference

Adaptive Blind All-in-One Image Restoration

Toward Efficient Deep Blind RAW Image Restoration

Diffusion Models for Image Restoration and Enhancement - A Comprehensive Survey

Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration

Step-Calibrated Diffusion for Biomedical Optical Image Restoration

Inversion by Direct Iteration: An Alternative to Denoising Diffusion for Image Restoration

MRIR: Integrating Multimodal Insights for Diffusion-based Realistic Image Restoration