Abstract:Many applications have benefited from the images with both high spatial and spectral resolution, such as mineralogy and surveillance. However, it is difficult to acquire such images due to the limitation of sensor technologies. Recently, super-resolution (SR) techniques have been proposed to improve the spatial or spectral resolution of images, e.g., improving the spatial resolution of hyperspectral images (HSIs) or improving spectral resolution of color images (reconstructing HSIs from RGB inputs). However, none of the researches attempted to improve both spatial and spectral resolution together. In this article, these two types of resolution are jointly improved using convolutional neural network (CNN). Specifically, two kinds of CNN-based SR are conducted, including a simultaneous spatial-spectral joint SR (SimSSJSR) that conducts SR in spectral and spatial domain simultaneously and a separated spatial-spectral joint SR (SepSSJSR) that considers spectral and spatial SR sequentially. In the proposed SimSSJSR, a full 3-D CNN is constructed to learn an end-to-end mapping between a low spatial-resolution mulitspectral image (LR-MSI) and the corresponding high spatial-resolution HSI (HR-HSI). In the proposed SepSSJSR, a spatial SR network and a spectral SR network are designed separately, and thus two different frameworks are proposed for SepSSJSR, namely SepSSJSR1 and SepSSJSR2, according to the order that spatial SR and spectral SR are applied. Furthermore, the least absolute deviation, instead of mean square error (MSE) in traditional SR networks, is chosen as the loss function for the proposed networks. Experimental results over simulated images from different sensors demonstrated that the proposed SepSSJSR1 is most effective to improve spatial and spectral resolution of MSIs sequentially by conducting spatial SR prior to spectral SR. In addition, validation on real Landsat images also indicates that the proposed SSJSR techniques can make full use of available MSIs for high-resolution-based analysis or applications.

Joint learning for single-image super-resolution via a coupled constraint.

Image Super-Resolution with Sparse Neighbor Embedding

Single Image Super-Resolution Via Locally Regularized Anchored Neighborhood Regression and Nonlocal Means.

Learning Super-Resolution Jointly from External and Internal Examples

Image Super-Resolution Based on Adaptive Joint Distribution Modeling

Image Super-resolution Based on Sparse Representation With Joint Constraints

Learning-based Super-Resolution Method with a Combining of Both Global and Local Constraints

Joint Sub-Band Based Neighbor Embedding for Image Super-Resolution

Joint Learning of Multiple Regressors for Single Image Super-Resolution

Spatial and Spectral Joint Super-Resolution Using Convolutional Neural Network.

Single Remote Sensing Image Super-Resolution with an Adaptive Joint Constraint Model

Single Image Super-Resolution with Multiscale Similarity Learning.

Learning super resolution with global and local constraints

Super Resolution Reconstruction via Multiple Frames Joint Learning

Single Image Super-Resolution With Learning Iteratively Non-Linear Mapping Between Low- and High-Resolution Sparse Representations

Learning Local Dictionaries and Similarity Structures for Single Image Super-Resolution.

Learning Coupled Dictionaries from Unpaired Data for Image Super-Resolution

Single-Image Superresolution Based on Local Regression and Nonlocal Self-Similarity

Learning Recurrent Residual Regressors for Single Image Super-Resolution

Single-Image Super-Resolution Via Adaptive Joint Kernel Regression.

Single Image Super-Resolution Based on Nonlocal Similarity and Sparse Representation