Abstract:Hyperspectral imaging is a promising imaging modality that simultaneously captures several images for the same scene on narrow spectral bands, and it has made considerable progress in different fields, such as agriculture, astronomy, and surveillance. However, the existing hyperspectral (HS) cameras sacrifice the spatial resolution for providing the detail spectral distribution of the imaged scene, which leads to low-resolution (LR) HS images compared with the common red-green-blue (RGB) images. Generating a high-resolution HS (HR-HS) image via fusing an observed LR-HS image with the corresponding HR-RGB image has been actively studied. Existing methods for this fusing task generally investigate hand-crafted priors to model the inherent structure of the latent HR-HS image, and they employ optimization approaches for solving it. However, proper priors for different scenes can possibly be diverse, and to figure it out for a specific scene is difficult. This study investigates a deep convolutional neural network (DCNN)-based method for automatic prior learning, and it proposes a novel fusion DCNN model with multi-scale spatial and spectral learning for effectively merging an HR-RGB and LR-HS images. Specifically, we construct an U-shape network architecture for gradually reducing the feature sizes of the HR-RGB image (Encoder-side) and increasing the feature sizes of the LR-HS image (Decoder-side), and we fuse the HR spatial structure and the detail spectral attribute in multiple scales for tackling the large resolution difference in spatial domain of the observed HR-RGB and LR-HS images. Then, we employ multi-level cost functions for the proposed multi-scale learning network to alleviate the gradient vanish problem in long-propagation procedure. In addition, for further improving the reconstruction performance of the HR-HS image, we refine the predicted HR-HS image using an alternating back-projection method for minimizing the reconstruction errors of the observed LR-HS and HR-RGB images. Experiments on three benchmark HS image datasets demonstrate the superiority of the proposed method in both quantitative values and visual qualities.

Hsvcnn: Cnn-Based Hyperspectral Reconstruction From Rgb Videos

HSCNN+: Advanced CNN-Based Hyperspectral Recovery from RGB Images

Compressive Single-Pixel Hyperspectral Imaging Using RGB Sensors

Fast non-iterative blind restoration of hyperspectral images with PSFs

Residual HSRCNN: Residual Hyper-Spectral Reconstruction CNN from an RGB Image

Hyperspectral image recovery based on fusion of coded aperture snapshot spectral imaging and RGB images by guided filtering

HSCNN: CNN-Based Hyperspectral Image Recovery from Spectrally Undersampled Projections

Deep-learning-based hyperspectral imaging through a RGB camera

Cluster-based deep convolutional networks for spectral reconstruction from RGB images

RGB-to-HSV: A Frequency-Spectrum Unfolding Network for Spectral Super-Resolution of RGB Videos

Random Convolutional Network for Hyperspectral Image Classification.

Multispectral Fusion of RGB and NIR Images Using Weighted Least Squares and Convolution Neural Networks

Hyperspectral Image Reconstruction Using Multi-scale Fusion Learning

Joint learning of RGBW color filter arrays and demosaicking

Hyperspectral Reconstruction From RGB Images via Physically Guided Graph Deep Prior Learning

SSF-CNN: Spatial and Spectral Fusion with CNN for Hyperspectral Image Super-Resolution

Spectral Image Reconstruction Using Recovered Basis Vector Coefficients

Spectral Super-Resolution via Model-Guided Cross-Fusion Network

Data-driven framework for high-accuracy color restoration of RGBN multispectral filter array sensors under extremely low-light conditions.

Universal high spatial resolution hyperspectral imaging using hybrid-resolution image fusion

Coarse-Fine Spectral-Aware Deformable Convolution For Hyperspectral Image Reconstruction