Spatial-Frequency Domain Information Integration for Pan-Sharpening

Man Zhou,Jie Huang,Keyu Yan,Hu Yu,Xueyang Fu,Aiping Liu,Xian Wei,Feng Zhao

DOI: https://doi.org/10.1007/978-3-031-19797-0_16

2022-01-01

Abstract:Pan-sharpening aims to generate high-resolution multispectral (MS) images by fusing PAN images and low-resolution MS images. Despite its great advances, most existing pan-sharpening methods only work in the spatial domain and rarely explore the potential solutions in the frequency domain. In this paper, we first attempt to address pan-sharpening in both spatial and frequency domains and propose a Spatial-Frequency Information Integration Network, dubbed as SFIIN. To implement SFIIN, we devise a core building module tailored with pansharpening, consisting of three key components: spatial-domain information branch, frequency-domain information branch, and dual domain interaction. To be specific, the first employs the standard convolution to integrate the local information of two modalities of PAN and MS images in the spatial domain, while the second adopts deep Fourier transformation to achieve the image-wide receptive field for exploring global contextual information. Followed by, the third is responsible for facilitating the information flow and learning the complementary representation. We conduct extensive experiments to validate the effectiveness of the proposed network and demonstrate the favorable performance against other state-of-the-art methods.