Progressive Pan-Sharpening Via Cross-Scale Collaboration Networks

Zihe Yang,Xueyang Fu,Aiping Liu,Zheng-Jun Zha

DOI: https://doi.org/10.1109/lgrs.2022.3170376

IF: 5.343

2022-01-01

IEEE Geoscience and Remote Sensing Letters

Abstract:Pan-sharpening aims to produce a high-quality image by fusing a low-resolution multispectral (LRMS) image and a high-resolution panchromatic (PAN) image. Although deep-learning-based methods have dominated the pan-sharpening, they fail to fully utilize spatial and spectral information from the cross-scale perspective. In this work, we propose a novel cross-scale collaboration network to achieve accurate pan-sharpening. Specifically, we first design a progressive framework in a pyramid fashion to achieve a gradual pan-sharpening process, which consists of several subnetworks to handle specific pyramid levels. Then, in each subnetwork, we deploy two cross-scale attention modules to, respectively, capture global and local spatial interactions from MS and PAN images. To better utilize spectral information from different pyramid levels, we further deploy a fusion module between subnetworks to extract cross-scale representations. Through the above-mentioned cross-scale collaboration, our network can fully consider the cross-scale nature of MS and PAN images to better adapt to this specific task. Extensive experiments confirmed that our method outperforms several state-of-the-art (SOTA) methods both qualitatively and quantitatively.