Learning Dynamic Generative Attention for Single Image Super-Resolution

Rui Chen,Yan Zhang

DOI: https://doi.org/10.1109/tcsvt.2022.3192099

IF: 5.859

2022-12-10

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Attention mechanisms have achieved great success for image super-resolution as they can effectively improve the feature representation ability. However, most attention-based methods produce the static attention weights, which are applied identically for all input samples. This popular attention strategy is difficult to automatically adapt the content variations of each individual input, hence hindering further improvements of the magnification performance. To explore towards resolving this challenge, we propose a variational hybrid network with newly dynamic attention mechanisms for image super-resolution tasks. Specifically, we design a multi-scale variational encoder network to transform the curvature map of an input image into the latent space. This is made possible for randomly generated latent variables to reflect the valuable high-frequency information and recalibrate the main network. We utilize these latent variables to further generate controllable attention weights, which modulate not only frequency parameters of convolutional kernels but also spatial characteristics of feature maps for boosting representation power. Moreover, a curvature-domain loss is designed to help the main network to concentrate more on high-frequency geometric structures. Experimental results have revealed that our method can generate more realistic and visually pleasing high-resolution images in comparison to state-of-the-art methods.

engineering, electrical & electronic