Abstract:Image deblurring continues to achieve impressive performance with the development of generative models. Nonetheless, there still remains a displeasing problem if one wants to improve perceptual quality and quantitative scores of recovered image at the same time. In this study, drawing inspiration from the research of transformer properties, we introduce the pretrained transformers to address this problem. In particular, we leverage deep features extracted from a pretrained vision transformer (ViT) to encourage recovered images to be sharp without sacrificing the performance measured by the quantitative metrics. The pretrained transformer can capture the global topological relations (i.e., self-similarity) of image, and we observe that the captured topological relationships about the sharp image will change when blur occurs. By comparing the transformer features between recovered image and target one, the pretrained transformer provides high-resolution blur-sensitive semantic information, which is critical in measuring the sharpness of the deblurred image. On the basis of the advantages, we present two types of novel perceptual losses to guide image deblurring. One regards the features as vectors and computes the discrepancy between representations extracted from recovered image and target one in Euclidean space. The other type considers the features extracted from an image as a distribution and compares the distribution discrepancy between recovered image and target one. We demonstrate the effectiveness of transformer properties in improving the perceptual quality while not sacrificing the quantitative scores peak signal-to-noise ratio (PSNR) over the most competitive models, such as Uformer, Restormer, and NAFNet, on defocus deblurring and motion deblurring tasks. The code is available at https://github. com/erfect2020/TransformerPerceptualLoss.

BokehOrNot: Transforming Bokeh Effect with Image Transformer and Lens Metadata Embedding

Lens-to-Lens Bokeh Effect Transformation. NTIRE 2023 Challenge Report.

Selective Bokeh Effect Transformation.

Bokeh Rendering Based on Adaptive Depth Calibration Network

Variable Aperture Bokeh Rendering via Customized Focal Plane Guidance

BokehMe++: Harmonious Fusion of Classical and Neural Rendering for Versatile Bokeh Creation

Dr.Bokeh: DiffeRentiable Occlusion-aware Bokeh Rendering

Video Bokeh Rendering: Make Casual Videography Cinematic

Ground-to-air Aircraft Infrared Image Deblurring Based on Imaging Degradation Simulation

Natural & Adversarial Bokeh Rendering via Circle-of-Confusion Predictive Network

Feature Pyramid Boosting Network for Rendering Natural Bokeh

BokehMe: When Neural Rendering Meets Classical Rendering

Constrained Predictive Filters for Single Image Bokeh Rendering.

NAFBET: Bokeh Effect Transformation with Parameter Analysis Block Based on NAFNet.

AdvBokeh: Learning to Adversarially Defocus Blur

Depth-guided Deep Filtering Network for Efficient Single Image Bokeh Rendering

Defocus to Focus: Photo-realistic Bokeh Rendering by Fusing Defocus and Radiance Priors

Hybrid Neural Rendering for Large-Scale Scenes with Motion Blur

AIM 2020 Challenge on Rendering Realistic Bokeh

Interactive Portrait Bokeh Rendering System

Image Deblurring by Exploring In-Depth Properties of Transformer