Abstract:Recently, convolutional neural networks (CNN) have obtained promising results in single-image SR for hyperspectral pansharpening. However, enhancing CNNs' representation ability with fewer parameters and a shorter prediction time is a challenging and critical task. In this paper, we propose a novel multi-spectral image fusion method using a combination of the previously proposed 3D CNN model VolumeNet and 2.5D texture transfer method using other modality high resolution (HR) images. Since a multi-spectral (MS) image consists of several bands and each band is a 2D image slice, MS images can be seen as 3D data. Thus, we use the previously proposed VolumeNet to fuse HR panchromatic (PAN) images and bicubic interpolated MS images. Because the proposed 3D VolumeNet can effectively improve the accuracy by expanding the receptive field of the model, and due to its lightweight structure, we can achieve better performance against the existing method without purchasing a large number of remote sensing images for training. In addition, VolumeNet can restore the high-frequency information lost in the HR MR image as much as possible, reducing the difficulty of feature extraction in the following step: 2.5D texture transfer. As one of the latest technologies, deep learning-based texture transfer has been demonstrated to effectively and efficiently improve the visual performance and quality evaluation indicators of image reconstruction. Different from the texture transfer processing of RGB image, we use HR PAN images as the reference images and perform texture transfer for each frequency band of MS images, which is named 2.5D texture transfer. The experimental results show that the proposed method outperforms the existing methods in terms of objective accuracy assessment, method efficiency, and visual subjective evaluation.

Pansharpening Using Guided Filtering to Improve the Spatial Clarity of VHR Satellite Imagery

Pansharpening by Combining Enhanced Spatial-Spectral Fidelity and Gradient-Domain Guided Filtering

Efficient and Accurate Hyperspectral Pansharpening Using 3D VolumeNet and 2.5D Texture Transfer

A pansharpening method combining iterative filtering and NSST-NSML-PAPCNN to optimize spatial detail extraction and injection

Pansharpening Based on Fuzzy Logic and Edge Activity

Object-Based Area-to-Point Regression Kriging for Pansharpening

Pan-sharpening Based on Common Saliency Feature Analysis and Multiscale Spatial Information Extraction for Multiple Remote Sensing Images

Improving Gram–Schmidt Adaptive Pansharpening Method Using Support Vector Regression and Markov Random Field

Variational Pansharpening Based on Coefficient Estimation With Nonlocal Regression

Log-Gabor directional region entropy adaptive guided filtering for multispectral pansharpening

Spectral Fidelity and Spatial Enhancement: An Assessment and Cascading of Pan-Sharpening Techniques for Satellite Imagery

Pan-Sharpening via Multiscale Dynamic Convolutional Neural Network

A Multi-Stage Progressive Pansharpening Network Based on Detail Injection with Redundancy Reduction

PAN-Guided Band-Aware Multi-Spectral Feature Enhancement for Pan-Sharpening

Implementation and comparative quantitative assessment of different multispectral image pansharpening approches

Detail Information Prior Net for Remote Sensing Image Pansharpening

A review of image fusion techniques for pan-sharpening of high-resolution satellite imagery

Hyperspectral Pansharpening: A Review

An IHS-Based Pan-Sharpening Method for Spectral Fidelity Improvement Using Ripplet Transform and Compressed Sensing

An Improved Version of the Generalized Laplacian Pyramid Algorithm for Pansharpening

Pansharpening of WorldView-2 Data via Graph Regularized Sparse Coding and Adaptive Coupled Dictionary