Abstract:In this paper, we address the unified image restoration challenge by reframing it as a contrastive learning-based classification problem. Despite the significant strides made by deep learning methods in enhancing image restoration quality, their limited capacity to generalize across diverse degradation types and intensities necessitates the training of separate models for each specific degradation scenario. We proposes an all-encompassing approach that can restore images from various unknown corruption types and levels. We devise a method that learns representations of the latent sharp image’s degradation and accompanying textual features (such as dataset categories and image content descriptions), converting these into prompts which are then embedded within a reconstruction network model to enhance cross-database restoration performance. This culminates in a unified image reconstruction framework. The study involves two stages: In the first stage, we design a MultiContentNet that learns multi-modal features (MMFs) of the latent sharp image. This network encodes the visual degradation expressions and contextual text features into latent variables, thereby exerting a guided classification effect. Specifically, MultiContentNet is trained as an auxiliary controller capable of taking the degraded input image and, through contrastive learning, extracts MMFs of the latent target image. This effectively generates natural classifiers tailored for different degradation types. The second phase integrates the learned MMFs into an image restoration network via cross-attention mechanisms. This guides the restoration model to learn high-fidelity image recovery. Experiments conducted on six blind image restoration tasks demonstrate that the proposed method achieves state-of-the-art performance, highlighting the potential significance of large-scale pretrained vision-language models’ MMFs in advancing high-quality unified image reconstruction.

Deep Multi-Label Learning for Image Distortion Identification

Multi-modal Degradation Feature Learning for Unified Image Restoration Based on Contrastive Learning

Systemic distortion analysis with deep distortion directed image quality assessment models

Distortion Robust Image Classification using Deep Convolutional Neural Network with Discrete Cosine Transform

Multi-Task Rank Learning for Image Quality Assessment

MLMA-Net: multi-level multi-attentional learning for multi-label object detection in textile defect images

Multi-label remote sensing image classification with deformable convolutions and graph neural networks

Multi-scale and Discriminative Part Detectors Based Features for Multi-label Image Classification.

Dynamic Correlation Learning and Regularization for Multi-Label Confidence Calibration

CTransCNN: Combining transformer and CNN in multilabel medical image classification

Asymmetric Vision Transformers for Multi-Label Classification

Dual-stream multi-label image classification model enhanced by feature reconstruction

Midcn: A Multiple Instance Deep Convolutional Network For Image Classification

Learning Local Instance Constraint For Multi-Label Classification

CNN: Single-label to Multi-label

Rate Distortion Multiple Instance Learning for Image Classification

Multi-task learning based on geometric invariance discriminative features

A novel plant type, leaf disease and severity identification framework using CNN and transformer with multi-label method

Reconstruction Regularized Deep Metric Learning for Multi-label Image Classification

Learning to Detect Multiple Photographic Defects