Abstract:In low-level vision such as single image super-resolution (SISR), traditional MSE or L-1 loss function treats every pixel equally with the assumption that the importance of all pixels is the same. However, it has been long recognized that texture and edge areas carry more important visual information than smooth areas in photographic images. How to achieve such spatial adaptation in a principled manner has been an open problem in both traditional model-based and modern learning-based approaches toward SISR. In this paper, we propose a new adaptive weighted loss for SISR to train deep networks focusing on challenging situations such as textured and edge pixels with high uncertainty. Specifically, we introduce variance estimation characterizing the uncertainty on a pixel-by-pixel basis into SISR solutions so the targeted pixels in a high-resolution image (mean) and their corresponding uncertainty (variance) can be learned simultaneously. Moreover, uncertainty estimation allows us to leverage conventional wisdom such as sparsity prior for regularizing SISR solutions. Ultimately, pixels with large certainty (e.g., texture and edge pixels) will be prioritized for SISR according to their importance to visual quality. For the first time, we demonstrate that such uncertainty-driven loss can achieve better results than MSE or L-1 loss for a wide range of network architectures. Experimental results on three popular SISR networks show that our proposed uncertainty-driven loss has achieved better PSNR performance than traditional loss functions without any increased computation during testing. The code is available at https://see.xidian.edu.cn/faculty/wsdong/Projects/UDL-SR.htm

A Novel Perceptual Loss Function for Single Image Super-Resolution.

Adaptive Loss Function for Super Resolution Neural Networks Using Convex Optimization Techniques

Single Image Super-Resolution Via Perceptual Loss Guided By Denoising Auto-Encoder

Dual Perceptual Loss for Single Image Super-Resolution Using ESRGAN

Uncertainty-Driven Loss for Single Image Super-Resolution

Orientation-Aware Deep Neural Network for Real Image Super-Resolution.

Learning Structral Coherence Via Generative Adversarial Network for Single Image Super-Resolution

Fast and Accurate Single Image Super-Resolution Via an Energy-Aware Improved Deep Residual Network.

Generative Adversarial Network-based Image Super-Resolution with a Novel Quality Loss

Boundary Equilibrium SR: Effective Loss Functions for Single Image Super-Resolution

Non-local Network Routing for Perceptual Image Super-Resolution

Progressive perception-oriented network for single image super-resolution

Perceptually Optimized Super Resolution

Deep Objective Quality Assessment Driven Single Image Super-Resolution

Deep Feature Translation Network Guided by Combined Loss for Single Image Super-Resolution

Perception-oriented Single Image Super-Resolution Network with Receptive Field Block

A Novel Multiconnected Convolutional Network for Super-Resolution.

Learning from Multi-Perception Features for Real-Word Image Super-resolution

Non-linear Perceptual Multi-Scale Network for Single Image Super-Resolution

SRPGAN: Perceptual Generative Adversarial Network for Single Image Super Resolution.

Joint Learning of Super-Resolution and Perceptual Image Enhancement for Single Image.