Abstract:In this paper, we address the unified image restoration challenge by reframing it as a contrastive learning-based classification problem. Despite the significant strides made by deep learning methods in enhancing image restoration quality, their limited capacity to generalize across diverse degradation types and intensities necessitates the training of separate models for each specific degradation scenario. We proposes an all-encompassing approach that can restore images from various unknown corruption types and levels. We devise a method that learns representations of the latent sharp image’s degradation and accompanying textual features (such as dataset categories and image content descriptions), converting these into prompts which are then embedded within a reconstruction network model to enhance cross-database restoration performance. This culminates in a unified image reconstruction framework. The study involves two stages: In the first stage, we design a MultiContentNet that learns multi-modal features (MMFs) of the latent sharp image. This network encodes the visual degradation expressions and contextual text features into latent variables, thereby exerting a guided classification effect. Specifically, MultiContentNet is trained as an auxiliary controller capable of taking the degraded input image and, through contrastive learning, extracts MMFs of the latent target image. This effectively generates natural classifiers tailored for different degradation types. The second phase integrates the learned MMFs into an image restoration network via cross-attention mechanisms. This guides the restoration model to learn high-fidelity image recovery. Experiments conducted on six blind image restoration tasks demonstrate that the proposed method achieves state-of-the-art performance, highlighting the potential significance of large-scale pretrained vision-language models’ MMFs in advancing high-quality unified image reconstruction.

Image Quality Assessment–driven Reinforcement Learning for Mixed Distorted Image Restoration

Mixed distortion image enhancement method based on joint of deep residuals learning and reinforcement learning

Boosting denoisers with reinforcement learning for image restoration

In-training Restoration Models Matter: Data Augmentation for Degraded-reference Image Quality Assessment

In-training Restoration Models Matter

JE2NET: Joint Exploitation and Exploration in Reinforcement Learning Based Image Restoration

Mixed Degradation Image Restoration via Local Dynamic Optimization and Conditional Embedding

Deep Learning-based Distortion Sensitivity Prediction for Full-Reference Image Quality Assessment

Multi-modal Degradation Feature Learning for Unified Image Restoration Based on Contrastive Learning

Visual Recognition-Driven Image Restoration for Multiple Degradation with Intrinsic Semantics Recovery

Accuracy and generalization improvement for image quality assessment of authentic distortion by semi-supervised learning

A Comprehensive Review of Deep Learning-Based Real-World Image Restoration

Joint Distortion Restoration and Quality Feature Learning for No-Reference Image Quality Assessment

No-Reference Image Quality Assessment: an Attention Driven Approach

Deep Likelihood Network for Image Restoration with Multiple Degradation Levels

Joint Deep Image Restoration and Unsupervised Quality Assessment

Distort-and-Recover: Color Enhancement using Deep Reinforcement Learning

Image Quality Assessment Guided Deep Neural Networks Training

Integrating Degradation Learning into Image Restoration

All-in-one Multi-degradation Image Restoration Network via Hierarchical Degradation Representation

Low-quality Image Object Detection Based on Reinforcement Learning Adaptive Enhancement