Abstract:Many applications have benefited from the images with both high spatial and spectral resolution, such as mineralogy and surveillance. However, it is difficult to acquire such images due to the limitation of sensor technologies. Recently, super-resolution (SR) techniques have been proposed to improve the spatial or spectral resolution of images, e.g., improving the spatial resolution of hyperspectral images (HSIs) or improving spectral resolution of color images (reconstructing HSIs from RGB inputs). However, none of the researches attempted to improve both spatial and spectral resolution together. In this article, these two types of resolution are jointly improved using convolutional neural network (CNN). Specifically, two kinds of CNN-based SR are conducted, including a simultaneous spatial-spectral joint SR (SimSSJSR) that conducts SR in spectral and spatial domain simultaneously and a separated spatial-spectral joint SR (SepSSJSR) that considers spectral and spatial SR sequentially. In the proposed SimSSJSR, a full 3-D CNN is constructed to learn an end-to-end mapping between a low spatial-resolution mulitspectral image (LR-MSI) and the corresponding high spatial-resolution HSI (HR-HSI). In the proposed SepSSJSR, a spatial SR network and a spectral SR network are designed separately, and thus two different frameworks are proposed for SepSSJSR, namely SepSSJSR1 and SepSSJSR2, according to the order that spatial SR and spectral SR are applied. Furthermore, the least absolute deviation, instead of mean square error (MSE) in traditional SR networks, is chosen as the loss function for the proposed networks. Experimental results over simulated images from different sensors demonstrated that the proposed SepSSJSR1 is most effective to improve spatial and spectral resolution of MSIs sequentially by conducting spatial SR prior to spectral SR. In addition, validation on real Landsat images also indicates that the proposed SSJSR techniques can make full use of available MSIs for high-resolution-based analysis or applications.

Hyperspectral Image Super-Resolution Method Based on Spatial Spectral Joint Sparse Representation

Hyperspectral Image Super Resolution Reconstruction with a Joint Spectral-Spatial Sub-Pixel Mapping Model.

Spectral Representation Via Data-Guided Sparsity for Hyperspectral Image Super-Resolution

Hyperspectral imagery super-resolution by sparse representation and spectral regularization

Compressed Sensing Reconstruction Of Hyperspectral Images Jointly Using Spatial Smoothing Feature And Spectral Correlation

Hyperspectral Image Super-Resolution Through Clustering-Based Sparse Representation.

Hyperspectral Image Super-Resolution Based on Tensor Spatial-Spectral Joint Correlation Regularization

Hyperspectral Imagery Super-Resolution by Spatial–Spectral Joint Nonlocal Similarity

Hyperspectral Image Super-Resolution via Non-Negative Structured Sparse Representation

Hyper-Spectral Image Super-Resolution Using Non-Negative Spectral Representation with Data-Guided Sparsity

Hyperspectral Image Super-Resolution Via Convolutional Neural Network

Hyperspectral Image Super-Resolution by Spectral Mixture Analysis and Spatial–Spectral Group Sparsity

Hyperspectral Image Super-Resolution Based on Spatial and Spectral Correlation Fusion

Self-Similarity Constrained Sparse Representation for Hyperspectral Image Super-Resolution.

Joint Spatial-Spectral Resolution Enhancement Of Multispectral Images With Spectral Matrix Factorization And Spatial Sparsity Constraints

Spatial and Spectral Joint Super-Resolution Using Convolutional Neural Network.

Hyperspectral Superresolution Reconstruction Via Decomposition of Low-Rank and Sparse Tensor

Hyperspectral imagery super-resolution by image fusion and compressed sensing

A SPATIAL CONSTRAINT AND DEEP LEARNING BASED HYPERSPECTRAL IMAGE SUPER-RESOLUTION METHOD

Hyperspectral Images Reconstruction Based Super-Pixel Mapping Using Cross-Channel Sparse Model

Spatial-Spectral Joint Compressed Sensing for Hyperspectral Images