Abstract:With the development of edge intelligent devices such as unmanned aerial vehicles (UAVs), the demand of for high-resolution (HR) images increase significantly. However, noise and blurring from finite inspector sizes and optics make high-resolution images difficult to acquire directly. Therefore, lightweight super-resolution of optical images based on convolutional neural network (CNN) has become a hot spot. While most state-of-the-art methods pay more attention to local features on a particular dimension. Though composite attention mechanism has employed in them, the conflict among features of different attention types affects the SR performance thoroughly. In this paper, we propose a dynamic feature distillation and pyramid split large kernel attention network (DPLKA) to solve such problems. In particular, a pyramid split large kernel attention module (PSLKA) is introduced to obtain the multi-scale global information and long-range dependence. Subsequently, by constructing a global-to-local feature extraction block (GL-FEB), a global-to-local feature extraction approach similar to swin transformer with multi-scale self-attention is established. Furthermore, a dynamic feature distillation block (DFDB) is considered in this model with the purpose of utilizing hierarchical features from different layers and realizing adaptive recalibration of different response. Specifically, DPLKA applies lightweight architecture such as depth-wise separable convolution (SDC) and distillation feature extraction module (DFEM) which greatly improves the effectiveness of the method. Extensive experimental results on five benchmark datasets indicate that DPLKA is dominant in reconstruction accuracy (0.21 ~ 2 dB in Urban100 dataset with the scale of ), excellent running time (0.047 s in Urban100 dataset) and recipient parameters and flops.

Knowledge Distillation with Multi-granularity Mixture of Priors for Image Super-Resolution

MTKD: Multi-Teacher Knowledge Distillation for Image Super-Resolution

Self-Paced Knowledge Distillation for Real-Time Image Guided Depth Completion

Data Upcycling Knowledge Distillation for Image Super-Resolution

PKD-Net: Distillation of Prior Knowledge for Image Completion by Multi-Level Semantic Attention

Learning knowledge representation with meta knowledge distillation for single image super-resolution

Prior Knowledge Distillation Network for Face Super-Resolution

Knowledge Representing: Efficient, Sparse Representation of Prior Knowledge for Knowledge Distillation

Fakd: feature-affinity based knowledge distillation for efficient image super-resolution

Collaborative Knowledge Distillation Via Multiknowledge Transfer.

Pixel Distillation: A New Knowledge Distillation Scheme for Low-Resolution Image Recognition

Lightweight Self-Knowledge Distillation with Multi-source Information Fusion

Coupled image and kernel prior learning for high-generalized super-resolution

Large Kernel Distillation Network for Efficient Single Image Super-Resolution

CDFKD-MFS: Collaborative Data-free Knowledge Distillation Via Multi-level Feature Sharing

CDFKD-MFS: Collaborative Data-free Knowledge Distillation via Multi-level Feature Sharing

Knowledge Distillation with Feature Maps for Image Classification

Knowledge Distillation based Degradation Estimation for Blind Super-Resolution

An Embarrassingly Simple Approach for Knowledge Distillation

Online Knowledge Distillation via Collaborative Learning

Dynamic feature distillation and pyramid split large kernel attention network for lightweight image super-resolution