Abstract:Many applications have benefited from the images with both high spatial and spectral resolution, such as mineralogy and surveillance. However, it is difficult to acquire such images due to the limitation of sensor technologies. Recently, super-resolution (SR) techniques have been proposed to improve the spatial or spectral resolution of images, e.g., improving the spatial resolution of hyperspectral images (HSIs) or improving spectral resolution of color images (reconstructing HSIs from RGB inputs). However, none of the researches attempted to improve both spatial and spectral resolution together. In this article, these two types of resolution are jointly improved using convolutional neural network (CNN). Specifically, two kinds of CNN-based SR are conducted, including a simultaneous spatial-spectral joint SR (SimSSJSR) that conducts SR in spectral and spatial domain simultaneously and a separated spatial-spectral joint SR (SepSSJSR) that considers spectral and spatial SR sequentially. In the proposed SimSSJSR, a full 3-D CNN is constructed to learn an end-to-end mapping between a low spatial-resolution mulitspectral image (LR-MSI) and the corresponding high spatial-resolution HSI (HR-HSI). In the proposed SepSSJSR, a spatial SR network and a spectral SR network are designed separately, and thus two different frameworks are proposed for SepSSJSR, namely SepSSJSR1 and SepSSJSR2, according to the order that spatial SR and spectral SR are applied. Furthermore, the least absolute deviation, instead of mean square error (MSE) in traditional SR networks, is chosen as the loss function for the proposed networks. Experimental results over simulated images from different sensors demonstrated that the proposed SepSSJSR1 is most effective to improve spatial and spectral resolution of MSIs sequentially by conducting spatial SR prior to spectral SR. In addition, validation on real Landsat images also indicates that the proposed SSJSR techniques can make full use of available MSIs for high-resolution-based analysis or applications.

Stereoscopic Image Super-Resolution Method with View Incorporation and Convolutional Neural Networks

Spatial and Spectral Joint Super-Resolution Using Convolutional Neural Network.

Hyperspectral Image Super-Resolution Via Convolutional Neural Network

Stereo Endoscopic Image Super-Resolution Using Disparity-Constrained Parallel Attention

Orientation-Aware Deep Neural Network for Real Image Super-Resolution.

Video Super-Resolution Reconstruction Based on Deep Convolutional Neural Network and Spatio-Temporal Similarity

3D Video Super-Resolution Using Fully Convolutional Neural Networks

Sub-Pixel Convolutional Neural Network for Image Super-Resolution Reconstruction

Disparity-constrained Stereo Endoscopic Image Super-Resolution.

Image super-resolution using CNNs for dual-focal camera system

Hyperspectral Image Spatial Super-Resolution Via 3d Full Convolutional Neural Network

Image Super-Resolution Via a Novel Cascaded Convolutional Neural Network Framework.

Structure-preserving video super-resolution using three-dimensional convolutional neural networks

Stereo Video Super-Resolution Via Exploiting View-Temporal Correlations.

MR Image Super-Resolution via Wide Residual Networks With Fixed Skip Connection

Cross Parallax Attention Network for Stereo Image Super-Resolution.

SA-GNN: Stereo Attention and Graph Neural Network for Stereo Image Super-Resolution

Image Super-Resolution Using Deep Convolutional Networks

Coarse-to-Fine Image Super-Resolution Using Convolutional Neural Networks.

A New Convolutional Neural Network for Super-Resolution by Global and Local Residual

End-to-End Image Super-Resolution via Deep and Shallow Convolutional Networks