Abstract:Distortion identification and rectification in images and videos is vital for achieving good performance in downstream vision applications. Instead of relying on fixed trial-and-error based image processing pipelines, we propose a two-level sequential planning approach for automated image distortion classification and rectification. At the higher level it detects the class of corruptions present in the input image, if any. The lower level selects a specific algorithm to be applied, from a set of externally provided candidate algorithms. The entire two-level setup runs in the form of a single forward pass during inference and it is to be queried iteratively until the retrieval of the original image. We demonstrate improvements compared to three baselines on the object detection task on COCO image dataset with rich set of distortions. The advantage of our approach is its dynamic reconfiguration, conditioned on the input image and generalisability to unseen candidate algorithms at inference time, since it relies only on the comparison of their output of the image embeddings.

What problem does this paper attempt to address?

This paper attempts to solve different kinds of distortion problems introduced during the acquisition, processing, compression, transmission, storage and retrieval of images and videos. These problems can seriously affect the performance of downstream vision tasks (such as image classification, object detection and segmentation). Specifically, the paper aims to automate the process of image distortion recognition and correction algorithm selection to ensure image quality and improve the performance of downstream tasks. ### Main problems 1. **Distortion recognition**: How to automatically recognize the types of distortion present in the input image. 2. **Correction algorithm selection**: How to dynamically select an appropriate correction algorithm according to the recognized distortion type to restore image quality. ### Paper solutions The paper proposes a framework named **DeepClean**, which automatically performs image distortion classification and correction through a two - level sequential planning method: - **High - level**: Detect whether there is distortion in the input image and its type. - **Low - level**: Select a specific correction algorithm from a set of externally provided candidate algorithms. ### Framework features - **Automation**: Automatically configure the image processing pipeline without human intervention. - **Dynamic reconfiguration**: Dynamically adjust the correction algorithm according to the characteristics of the input image. - **Generalization ability**: Be able to adapt to unseen candidate algorithms at inference time. - **Single - forward pass**: The entire two - layer setup only requires one forward pass during the inference process and can be iteratively queried until the original image is restored. ### Experimental verification The paper conducted experiments on the COCO dataset and demonstrated the superiority of DeepClean in object detection tasks, especially when dealing with rich distortion. Compared with three baseline methods, DeepClean showed significant improvement. ### Mathematical representation To describe the problem more clearly, the paper defines a quadruple \( \langle I, S, A, Q \rangle \): - \( I=\{i_1, i_2, \ldots\} \) is the set of input images. - \( S = \{s_1, s_2, \ldots\} \) is the set of image processing steps. - \( A=\{a_1, a_2, \ldots\} \) is the set of image processing algorithms. - \( Q: S\times A\rightarrow\mathbb{R} \) is a function that returns the quality of the solution obtained when the image processing step \( s_i\in S \) and the corresponding algorithm \( a_i\in A \) are selected to process the input image \( i_i\in I \). Learn a mapping function \( \pi: I\rightarrow\langle S, A \rangle \) to map each image \( i_i\in I \) to the image processing step \( s_i\in S \) and the algorithm \( a_i\in A \). ### Summary The DeepClean framework improves the efficiency and accuracy of the image processing pipeline by automating distortion recognition and correction algorithm selection, thereby enhancing the performance of downstream vision tasks.

DeepClean: Integrated Distortion Identification and Algorithm Selection for Rectifying Image Corruptions

Image restoration for real-world under-display imaging

DeepCorrect: Correcting DNN Models Against Image Distortions

A Deep Ordinal Distortion Estimation Approach for Distortion Rectification

An efficient deep learning-based framework for image distortion correction

Distortion Robust Image Classification using Deep Convolutional Neural Network with Discrete Cosine Transform

DaliID: Distortion-Adaptive Learned Invariance for Identification—A Robust Technique for Face Recognition and Person Re-Identification

FixPix: Fixing Bad Pixels using Deep Learning

Single Image Automatic Radial Distortion Compensation Using Deep Convolutional Network

OIDC-Net: Omnidirectional Image Distortion Correction via Coarse-to-Fine Region Attention

Detection of Image Tampering Using Deep Learning, Error Levels and Noise Residuals

Blind Geometric Distortion Correction on Images Through Deep Learning

A deep learning framework for quality assessment and restoration in video endoscopy

Encoding laparoscopic image to words using vision transformer for distortion classification and ranking in laparoscopic videos

Systemic distortion analysis with deep distortion directed image quality assessment models

Learning to Restore Multiple Image Degradations Simultaneously

Attention-based Adaptive Selection of Operations for Image Restoration in the Presence of Unknown Combined Distortions

Deep Image Restoration For Image Anti-Forensics

Deep Learning-based Distortion Sensitivity Prediction for Full-Reference Image Quality Assessment

Scrupulous SCGAN Framework for Recognition of Restored Images with Caffe based PCA Filtration

Dformer: Learning Efficient Image Restoration with Perceptual Guidance