RGTGAN: Reference-Based Gradient-Assisted Texture-Enhancement GAN for Remote Sensing Super-Resolution
Ziming Tu,Xiubin Yang,Xi He,Jiapu Yan,Tingting Xu
DOI: https://doi.org/10.1109/tgrs.2024.3359095
IF: 8.2
2024-02-10
IEEE Transactions on Geoscience and Remote Sensing
Abstract:Reference (Ref)-based super-resolution (Ref-SR) is a heated topic distinguished from single-image super-resolution (SISR). It aims at transferring more texture details from the Ref image with a different perspective to super-resolve the low-resolution (LR) image. However, the development of Ref-SR within the remote sensing (RS) community is limited by three problems. First, RS images exhibit more complex texture details compared to natural images. It is challenging to learn and reconstruct the fine texture of RS images. Second, the lack of a high-quality RS image dataset, which contains massive RS image pairs from different perspectives, hampers the model training and diminishes the generalization of Ref-SR within the RS community. Third, the lack of a physical system prevents it from verifying the feasibility of Ref-SR in RS practice. To address these problems, this article proposes a novel Ref-based gradient-assisted texture-enhancement generative adversarial network (RGTGAN), a novel dataset, namely, KaggleSRD, and a novel physical simulation system, namely, dual-zoom-lens system (DZLS). Specifically, this article proposes a gradient-assisted texture-enhancement module (GTEM) to fully release the potential of the gradient branch to learn fine structures during the feature extraction process, a novel dense-intern deformable convolution (DIDConv) to boost the alignment effect between features from different image branches during the feature alignment process, and a novel dense-restore-residual (DRR) module to effectively transfer features. Extensive experimental results on both datasets, RRSSRD and KaggleSRD, demonstrate the superiority of the proposed method over state-of-the-art methods. Furthermore, DZLS verifies promising application prospects of the proposed method. Our code and dataset are publicly available at https://github.com/stdinR/RGTGAN.
imaging science & photographic technology,remote sensing,engineering, electrical & electronic,geochemistry & geophysics