Abstract:Pansharpening refers to fusing a low-resolution multispectral image (LRMS) and a high-resolution panchromatic (PAN) image to generate a high-resolution multispectral image (HRMS). Traditional pansharpening methods use a single pair of LRMS and PAN to generate HRMS at full resolution, but they fail to generate high-quality fused products due to the assumption of a (often inaccurate) linear relationship between the fused products. Convolutional neural network methods, i.e. , supervised and unsupervised learning approaches, can model any arbitrary non-linear relationship among data, but performing even worse than traditional methods when testing data are not consistent with training data. Moreover, supervised methods rely on simulating reduced resolution data for training causing information loss at full resolution. Unsupervised pansharpening suffers from distortion due to the lack of reference images and inaccuracy in the estimation of the degradation process. In this paper, we propose a zero-shot semi-supervised method for pansharpening (ZS-Pan), which only requires a single pair of PAN/LRMS images for training and testing networks combining both the pros of supervised and unsupervised methods. Facing with challenges of limited training data and no reference images, the ZS-Pan framework is built with a two-phase three-component model, i.e. , the reduced resolution supervised pre-training (RSP), the spatial degradation establishment (SDE), and the full resolution unsupervised generation (FUG) stages. Specifically, a special parameter initialization technique, a data augmentation strategy, and a non-linear degradation network are proposed to improve the representation ability of the network. In our experiments, we evaluate the performance of the proposed framework on different datasets using some state-of-the-art (SOTA) pansharpening approaches for comparison. Results show that our ZS-Pan outperforms these SOTA methods, both visually and quantitatively. Codes will be available after possible acceptance.

Spectral Constrained Residual Attention Network for Hyperspectral Pansharpening.

SSETPAN: Spatial-Spectral Enhanced Transformer Based Network for Pansharpening

A Deep-Shallow Fusion Network With Multidetail Extractor and Spectral Attention for Hyperspectral Pansharpening

Hybrid Attention Based Residual Network for Pansharpening

Attention-based dual residual network-based for multi-spectral pan-sharpening

Spectral-Fidelity Convolutional Neural Networks for Hyperspectral Pansharpening

HyperPNN: Hyperspectral Pansharpening Via Spectrally Predictive Convolutional Neural Networks

Rethinking Pan-Sharpening Via Spectral-Band Modulation

Hyperspectral Imagery Super-Resolution Based on Self-Calibrated Attention Residual Network.

Unsupervised Hyperspectral Pansharpening by Ratio Estimation and Residual Attention Network

Hyperspectral Pansharpening via Deep Detail Injection Network

DiTBN: Detail Injection-Based Two-Branch Network for Pansharpening of Remote Sensing Images

Multispectral Pansharpening Based on Multisequence Convolutional Recurrent Neural Network

SC-PNN: Saliency Cascade Convolutional Neural Network for Pansharpening

Unsupervised Pansharpening Method Using Residual Network with Spatial Texture Attention.

Detail Information Prior Net for Remote Sensing Image Pansharpening

Zero-shot semi-supervised learning for pansharpening

Pansharpening Via Triplet Attention Network with Information Interaction

Spatial and Spectral Extraction Network With Adaptive Feature Fusion for Pansharpening

A Dual-Branch Detail Extraction Network for Hyperspectral Pansharpening

Pansharpening via Unsupervised Convolutional Neural Networks