Abstract:Recently, convolutional neural networks (CNN) have obtained promising results in single-image SR for hyperspectral pansharpening. However, enhancing CNNs' representation ability with fewer parameters and a shorter prediction time is a challenging and critical task. In this paper, we propose a novel multi-spectral image fusion method using a combination of the previously proposed 3D CNN model VolumeNet and 2.5D texture transfer method using other modality high resolution (HR) images. Since a multi-spectral (MS) image consists of several bands and each band is a 2D image slice, MS images can be seen as 3D data. Thus, we use the previously proposed VolumeNet to fuse HR panchromatic (PAN) images and bicubic interpolated MS images. Because the proposed 3D VolumeNet can effectively improve the accuracy by expanding the receptive field of the model, and due to its lightweight structure, we can achieve better performance against the existing method without purchasing a large number of remote sensing images for training. In addition, VolumeNet can restore the high-frequency information lost in the HR MR image as much as possible, reducing the difficulty of feature extraction in the following step: 2.5D texture transfer. As one of the latest technologies, deep learning-based texture transfer has been demonstrated to effectively and efficiently improve the visual performance and quality evaluation indicators of image reconstruction. Different from the texture transfer processing of RGB image, we use HR PAN images as the reference images and perform texture transfer for each frequency band of MS images, which is named 2.5D texture transfer. The experimental results show that the proposed method outperforms the existing methods in terms of objective accuracy assessment, method efficiency, and visual subjective evaluation.

Constrained Optimization Guided Approach for Multispectral and Panchromatic Image Fusion

Efficient and Accurate Hyperspectral Pansharpening Using 3D VolumeNet and 2.5D Texture Transfer

Unaligned Hyperspectral Image Fusion Via Registration and Interpolation Modeling

DuCoFPan: Dual-Condition Flow-based Network for Pan-sharpening

Multi-Spectral and Panchromatic Image Fusion Based on His-Wavelet Transform and Mopso Algorithm

Enhancing multispectral imagery spatial resolution using optimized adaptive PCA and high-pass modulation

Fusion of Hyperspectral and Multispectral Images: A Novel Framework Based on Generalization of Pan-Sharpening Methods

MMCMOO: A Novel Multispectral Pansharpening Method

Multispectral and Panchromatic Image Fusion Based on Improved Bilateral Filter

Panchromatic and multispectral image fusion for remote sensing and earth observation: Concepts, taxonomy, literature review, evaluation methodologies and challenges ahead

A review of image fusion techniques for pan-sharpening of high-resolution satellite imagery

Transformer-Based Dual-Branch Multiscale Fusion Network for Pan-Sharpening Remote Sensing Images

Pansharpening Using IHS Method on Multi-sensor Data and Multiple Feature Extraction Using Modified Otsu Thresholding

SSCAConv: Self-Guided Spatial-Channel Adaptive Convolution for Image Fusion

Asymmetric Bidirectional Fusion Network for Remote Sensing Pansharpening

Multispectral and Panchromatic Image Fusion Based on Unsubsampled Contourlet Transform

Multiscale Spatial-Frequency Domain Dynamic Pansharpening of Remote Sensing Images Integrated With Wavelet Transform

Hyperspectral and multispectral image fusion techniques for high resolution applications: a review

A pixel-level fusion method for multi-source optical remote sensing image combining the principal component analysis and curvelet transform

Multispectral Pan-sharpening via Dual-Channel Convolutional Network with Convolutional LSTM Based Hierarchical Spatial-Spectral Feature Fusion

Pansharpening by Combining Enhanced Spatial-Spectral Fidelity and Gradient-Domain Guided Filtering