PromptIR: Prompting for All-in-One Blind Image Restoration

Vaishnav Potlapalli,Syed Waqas Zamir,Salman Khan,Fahad Shahbaz Khan

2023-06-23

Abstract:Image restoration involves recovering a high-quality clean image from its degraded version. Deep learning-based methods have significantly improved image restoration performance, however, they have limited generalization ability to different degradation types and levels. This restricts their real-world application since it requires training individual models for each specific degradation and knowing the input degradation type to apply the relevant model. We present a prompt-based learning approach, PromptIR, for All-In-One image restoration that can effectively restore images from various types and levels of degradation. In particular, our method uses prompts to encode degradation-specific information, which is then used to dynamically guide the restoration network. This allows our method to generalize to different degradation types and levels, while still achieving state-of-the-art results on image denoising, deraining, and dehazing. Overall, PromptIR offers a generic and efficient plugin module with few lightweight prompts that can be used to restore images of various types and levels of degradation with no prior information on the corruptions present in the image. Our code and pretrained models are available here: <a class="link-external link-https" href="https://github.com/va1shn9v/PromptIR" rel="external noopener nofollow">this https URL</a>

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to address a key issue in the field of image restoration: how to construct a unified model that can effectively handle various types and degrees of image degradation. Traditional deep learning-based methods, while achieving significant results in specific types of image restoration tasks, often lack the generalization ability to different types and degrees of degradation. This means that for each different type of degradation (such as noise, rain streaks, haze, etc.), a separate model needs to be trained, and the degradation type of the input image needs to be known in advance when applying these models.

To overcome this limitation, the paper proposes a prompt-based learning method called PromptIR, which is a unified framework applicable to all types of image restoration. PromptIR encodes information about specific degradation types using prompts and utilizes these prompts to dynamically guide the restoration network, thereby achieving generalization to different degradation types. This approach allows the model to effectively restore high-quality clean images without requiring any prior information about the degradation type of the input image.

Specifically, the key contributions of PromptIR include:

1. **Proposing a prompt-based blind image restoration framework**: This framework relies solely on the input image itself to restore a clean image, without needing any prior knowledge about the degradation type.
2. **Designing a plug-in prompt module**: This module can be easily integrated into existing restoration networks and includes a Prompt Generation Module (PGM) and a Prompt Interaction Module (PIM). The PGM generates prompts related to the input conditions, while the PIM uses these prompts to dynamically adjust the input features to effectively remove degradation from the input image.
3. **Demonstrating the model's effectiveness and generalization ability through experiments**: PromptIR achieves state-of-the-art performance in various image restoration tasks (such as denoising, deraining, and dehazing), proving its strong capability as a unified model.

In summary, PromptIR provides a new solution for achieving an efficient and general image restoration model capable of handling various types of degradation.