Abstract:Hyperspectral imaging is a promising imaging modality that simultaneously captures several images for the same scene on narrow spectral bands, and it has made considerable progress in different fields, such as agriculture, astronomy, and surveillance. However, the existing hyperspectral (HS) cameras sacrifice the spatial resolution for providing the detail spectral distribution of the imaged scene, which leads to low-resolution (LR) HS images compared with the common red-green-blue (RGB) images. Generating a high-resolution HS (HR-HS) image via fusing an observed LR-HS image with the corresponding HR-RGB image has been actively studied. Existing methods for this fusing task generally investigate hand-crafted priors to model the inherent structure of the latent HR-HS image, and they employ optimization approaches for solving it. However, proper priors for different scenes can possibly be diverse, and to figure it out for a specific scene is difficult. This study investigates a deep convolutional neural network (DCNN)-based method for automatic prior learning, and it proposes a novel fusion DCNN model with multi-scale spatial and spectral learning for effectively merging an HR-RGB and LR-HS images. Specifically, we construct an U-shape network architecture for gradually reducing the feature sizes of the HR-RGB image (Encoder-side) and increasing the feature sizes of the LR-HS image (Decoder-side), and we fuse the HR spatial structure and the detail spectral attribute in multiple scales for tackling the large resolution difference in spatial domain of the observed HR-RGB and LR-HS images. Then, we employ multi-level cost functions for the proposed multi-scale learning network to alleviate the gradient vanish problem in long-propagation procedure. In addition, for further improving the reconstruction performance of the HR-HS image, we refine the predicted HR-HS image using an alternating back-projection method for minimizing the reconstruction errors of the observed LR-HS and HR-RGB images. Experiments on three benchmark HS image datasets demonstrate the superiority of the proposed method in both quantitative values and visual qualities.

Universal high spatial resolution hyperspectral imaging using hybrid-resolution image fusion

Multispectral Image Super-Resolution Via RGB Image Fusion and Radiometric Calibration

Unaligned Hyperspectral Image Fusion Via Registration and Interpolation Modeling

Curvelet Based Hyperspectral Image Fusion

Fusion of Hyperspectral-Multispectral images joining Spatial-Spectral Dual-Dictionary and structured sparse Low-rank representation

Hyperspectral Image Super Resolution With Real Unaligned RGB Guidance

Hyperspectral Image Reconstruction Using Multi-scale Fusion Learning

SSR-NET: Spatial–Spectral Reconstruction Network for Hyperspectral and Multispectral Image Fusion

Unsupervised Blind Spectral-spatial Cross-super-resolution Network for HSI and MSI Fusion

Hyperspectral and multispectral image fusion with arbitrary resolution through self-supervised representations

Boosting Image Super-Resolution Via Fusion of Complementary Information Captured by Multi-Modal Sensors

Image Fusion for Spatial Enhancement of Hyperspectral Image via Pixel Group Based Non-Local Sparse Representation

BUSIFusion: Blind Unsupervised Single Image Fusion of Hyperspectral and RGB Images

HMFT: Hyperspectral and Multispectral Image Fusion Super-Resolution Method Based on Efficient Transformer and Spatial-Spectral Attention Mechanism

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

Hyperspectral and Multispectral Image Fusion Based on Deep Attention Network.

Hyperspectral and multispectral image fusion via residual selective kernel attention-based U-net

Unified Cloud-Based Framework for Hyperspectral and Multispectral Image Fusion Incorporating Nonlocal Principles and Tensor Decomposition

Unsupervised Hybrid Network of Transformer and CNN for Blind Hyperspectral and Multispectral Image Fusion

Multispectral and hyperspectral images fusion based on subspace representation and nonlocal low-rank regularization

Fusion of Hyperspectral and Multispectral Images by Convolutional Sparse Representation