Abstract:Recently, convolutional neural networks (CNN) have obtained promising results in single-image SR for hyperspectral pansharpening. However, enhancing CNNs' representation ability with fewer parameters and a shorter prediction time is a challenging and critical task. In this paper, we propose a novel multi-spectral image fusion method using a combination of the previously proposed 3D CNN model VolumeNet and 2.5D texture transfer method using other modality high resolution (HR) images. Since a multi-spectral (MS) image consists of several bands and each band is a 2D image slice, MS images can be seen as 3D data. Thus, we use the previously proposed VolumeNet to fuse HR panchromatic (PAN) images and bicubic interpolated MS images. Because the proposed 3D VolumeNet can effectively improve the accuracy by expanding the receptive field of the model, and due to its lightweight structure, we can achieve better performance against the existing method without purchasing a large number of remote sensing images for training. In addition, VolumeNet can restore the high-frequency information lost in the HR MR image as much as possible, reducing the difficulty of feature extraction in the following step: 2.5D texture transfer. As one of the latest technologies, deep learning-based texture transfer has been demonstrated to effectively and efficiently improve the visual performance and quality evaluation indicators of image reconstruction. Different from the texture transfer processing of RGB image, we use HR PAN images as the reference images and perform texture transfer for each frequency band of MS images, which is named 2.5D texture transfer. The experimental results show that the proposed method outperforms the existing methods in terms of objective accuracy assessment, method efficiency, and visual subjective evaluation.

Cascaded Convolutional Neural Network-Based Hyperspectral Image Resolution Enhancement Via an Auxiliary Panchromatic Image.

Efficient and Accurate Hyperspectral Pansharpening Using 3D VolumeNet and 2.5D Texture Transfer

CNN-Based Hyperspectral Pansharpening With Arbitrary Resolution

Hyperspectral Image Super-Resolution Via Convolutional Neural Network

Novel Cross-Resolution Feature-Level Fusion for Joint Classification of Multispectral and Panchromatic Remote Sensing Images

Hyperspectral Image Super-Resolution with 1D-2D Attentional Convolutional Neural Network.

Feature Pyramid Fusion Network for Hyperspectral Pansharpening.

Coupled Spatial-Spectral Constrained Convolutional Fusion Network for Hyperspectral and Panchromatic images

HS-McHF: Hypersharpening With Multicomponent-Based Hierarchical Features Fusion Network

Going Deeper with Densely Connected Convolutional Neural Networks for Multispectral Pansharpening

Hyperspectral Image Super-Resolution Based on Spatial Correlation-Regularized Unmixing Convolutional Neural Network

A Multiscale and Multidepth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening.

Pansharpening Based on Multi-Branch CNN

A Spatio-Spectral Fusion Method for Hyperspectral Images Using Residual Hyper-Dense Network

Multispectral Pan-sharpening via Dual-Channel Convolutional Network with Convolutional LSTM Based Hierarchical Spatial-Spectral Feature Fusion

SSF-CNN: Spatial and Spectral Fusion with CNN for Hyperspectral Image Super-Resolution

Cross-Guided Pyramid Attention-Based Residual Hyperdense Network for Hyperspectral Image Pansharpening.

A Novel Unmixing-Based Hypersharpening Method via Convolutional Neural Network

Hyperspectral Image Super-Resolution Network Based on Cross-Scale Nonlocal Attention.

HyperPNN: Hyperspectral Pansharpening Via Spectrally Predictive Convolutional Neural Networks

Supervised-unsupervised Combined Deep Convolutional Neural Networks for High-Fidelity Pansharpening