See As You Desire: Scale-adaptive Face Super-Resolution for Varying Low Resolutions

Ling Li,Yan Zhang,Lin Yuan,Shuang Li,Xinbo Gao
DOI: https://doi.org/10.1109/jiot.2024.3492716
IF: 10.6
2024-01-01
IEEE Internet of Things Journal
Abstract:Face Super-Resolution (FSR) is critical for bolstering intelligent security in Internet of Things (IoT) systems. Recent deep learning-driven FSR algorithms have attained remarkable progress. However, they always require separate model training and optimization for each scaling factor or input resolution, leading to inefficiency and impracticality. To overcome these limitations, we propose SAFNet, an innovative framework tailored for scale-adaptive FSR with arbitrary input resolution. SAFNet integrates scale information into representation learning to enable adaptive feature extraction and introduces dual-embedding attention to boost adaptive feature reconstruction. It leverages facial self-similarity and spatial-frequency collaboration to achieve precise scale-aware SR representations. This is attained through three key modules: the Scale Adaption Guidance Unit (SAGU), the Scale-aware Non-Local Self-similarity (SNLS) module, and the Spatial-Frequency Interactive Modulation (SFIM) module. SAGU imports scaling factors using frequency encoding, SNLS exploits self-similarity to enrich feature representations, and SFIM incorporates spatial and frequency information to predict target pixel values adaptively. Comprehensive evaluations across four benchmark datasets reveal that SAFNet outperforms the second-best compared state-of-the-art (SOTA) method by about 0.2dB/0.007 in PSNR/SSIM (×4 on CelebA) with reduced 18.68%/42.64% computational complexity/time cost. This demonstrates SAFNet’s effectiveness and superiority, showcasing its potential as a promising solution for scale and input resolution adaptation challenges in FSR. The code will be available at https://github.com/ICVIPLab/SAFNethttps://github.com/ICVIPLab/SAFNet.
What problem does this paper attempt to address?