A Hybrid Non-Local Enhanced Network for Lightweight Single Image Super-Resolution

Zheng He,Haoran Yang,Lihui Chen,Gwanggil Jeon,Yiguang Liu,Xiaomin Yang

DOI: https://doi.org/10.1007/s00170-023-11913-0

2024-01-01

Abstract:In single image super-resolution (SISR) community, despite remarkable progress made by convolution neural network (CNN)-based methods, each convolution layer in networks merely incorporates feature dependencies in a limited neighborhood. Although deeply stacking the convolution layers can capture the long-range dependencies, it is implausible to be implemented in edge devices due to the huge computation burden. To address the above issues, we devise a hybrid non-local enhanced network (HNEN) with cost-effective non-local operations. Specifically, we develop a hybrid non-local unit (HNU) to harvest the long-range dependencies in both spatial and channel dimensions. For spatial dimension, the serial combination of row- and column-attention modules can incorporate the full-image dependencies with high computation efficiency. For channel dimension, the channel-attention module emphasizes the more correlative channels to further enhance the discriminative learning ability. Apart from HNU, we incorporate a multi-stage feature extraction block with a progressive learning strategy, fully exploiting the local hierarchical features and efficiently recovering the hard-to-cover information under resource-constrained circumstances. Extensive experimental results suggest that our method, with relatively low computational complexity and fewer network parameters, achieves comparable performance on all benchmarks against other state-of-the-art methods.