Abstract:Pansharpening refers to fusing a low-resolution multispectral image (LRMS) and a high-resolution panchromatic (PAN) image to generate a high-resolution multispectral image (HRMS). Traditional pansharpening methods use a single pair of LRMS and PAN to generate HRMS at full resolution, but they fail to generate high-quality fused products due to the assumption of a (often inaccurate) linear relationship between the fused products. Convolutional neural network methods, i.e. , supervised and unsupervised learning approaches, can model any arbitrary non-linear relationship among data, but performing even worse than traditional methods when testing data are not consistent with training data. Moreover, supervised methods rely on simulating reduced resolution data for training causing information loss at full resolution. Unsupervised pansharpening suffers from distortion due to the lack of reference images and inaccuracy in the estimation of the degradation process. In this paper, we propose a zero-shot semi-supervised method for pansharpening (ZS-Pan), which only requires a single pair of PAN/LRMS images for training and testing networks combining both the pros of supervised and unsupervised methods. Facing with challenges of limited training data and no reference images, the ZS-Pan framework is built with a two-phase three-component model, i.e. , the reduced resolution supervised pre-training (RSP), the spatial degradation establishment (SDE), and the full resolution unsupervised generation (FUG) stages. Specifically, a special parameter initialization technique, a data augmentation strategy, and a non-linear degradation network are proposed to improve the representation ability of the network. In our experiments, we evaluate the performance of the proposed framework on different datasets using some state-of-the-art (SOTA) pansharpening approaches for comparison. Results show that our ZS-Pan outperforms these SOTA methods, both visually and quantitatively. Codes will be available after possible acceptance.

A Review of Full Resolution Quality Assessment for Multispectral Pansharpening

Assessing Full-Resolution Pansharpening Quality: A Comparative Study of Methods and Measurements.

Efficient and Accurate Hyperspectral Pansharpening Using 3D VolumeNet and 2.5D Texture Transfer

Full-Resolution Quality Assessment for Pansharpening

Pansharpening by Combining Enhanced Spatial-Spectral Fidelity and Gradient-Domain Guided Filtering

Detail Information Prior Net for Remote Sensing Image Pansharpening

Rank Learning Based Full-Resolution Quality Evaluation Method for Pansharpened Images

Hyperspectral Pansharpening: A Review

Panchromatic and multispectral image fusion for remote sensing and earth observation: Concepts, taxonomy, literature review, evaluation methodologies and challenges ahead

Implementation and comparative quantitative assessment of different multispectral image pansharpening approches

Zero-shot semi-supervised learning for pansharpening

A Multi-Stage Progressive Pansharpening Network Based on Detail Injection with Redundancy Reduction

Variational Pansharpening Based on Coefficient Estimation With Nonlocal Regression

An Unsupervised CNN-Based Pansharpening Framework with Spectral-Spatial Fidelity Balance

Perceptual Quality Assessment of Pan-Sharpened Images

Unsupervised Pansharpening Based on Double-Cycle Consistency

A review of image fusion techniques for pan-sharpening of high-resolution satellite imagery

Multi-spectral Image Panchromatic Sharpening, Outcome and Process Quality Assessment Protocol

A Two-Stage Pansharpening Method for the Fusion of Remote-Sensing Images

Comprehensive Analysis and Improvements in Pansharpening Using Deep Learning

Improving Gram–Schmidt Adaptive Pansharpening Method Using Support Vector Regression and Markov Random Field