Abstract:Many applications have benefited from the images with both high spatial and spectral resolution, such as mineralogy and surveillance. However, it is difficult to acquire such images due to the limitation of sensor technologies. Recently, super-resolution (SR) techniques have been proposed to improve the spatial or spectral resolution of images, e.g., improving the spatial resolution of hyperspectral images (HSIs) or improving spectral resolution of color images (reconstructing HSIs from RGB inputs). However, none of the researches attempted to improve both spatial and spectral resolution together. In this article, these two types of resolution are jointly improved using convolutional neural network (CNN). Specifically, two kinds of CNN-based SR are conducted, including a simultaneous spatial-spectral joint SR (SimSSJSR) that conducts SR in spectral and spatial domain simultaneously and a separated spatial-spectral joint SR (SepSSJSR) that considers spectral and spatial SR sequentially. In the proposed SimSSJSR, a full 3-D CNN is constructed to learn an end-to-end mapping between a low spatial-resolution mulitspectral image (LR-MSI) and the corresponding high spatial-resolution HSI (HR-HSI). In the proposed SepSSJSR, a spatial SR network and a spectral SR network are designed separately, and thus two different frameworks are proposed for SepSSJSR, namely SepSSJSR1 and SepSSJSR2, according to the order that spatial SR and spectral SR are applied. Furthermore, the least absolute deviation, instead of mean square error (MSE) in traditional SR networks, is chosen as the loss function for the proposed networks. Experimental results over simulated images from different sensors demonstrated that the proposed SepSSJSR1 is most effective to improve spatial and spectral resolution of MSIs sequentially by conducting spatial SR prior to spectral SR. In addition, validation on real Landsat images also indicates that the proposed SSJSR techniques can make full use of available MSIs for high-resolution-based analysis or applications.

CASR-Net: A Color-Aware Super-Resolution Network for Panchromatic Image

Parallel-Connected Residual Channel Attention Network for Remote Sensing Image Super-Resolution

SCSNet: an Efficient Paradigm for Learning Simultaneously Image Colorization and Super-resolution

Hyperspectral Imagery Super-Resolution Based on Self-Calibrated Attention Residual Network.

CSRDNN: an Integrated Scheme for Single Satellite Image Colorization and Super-Resolution Using Deep Neural Networks.

A Deep Multitask Convolutional Neural Network for Remote Sensing Image Super-Resolution and Colorization

A Restoration Scheme for Spatial and Spectral Resolution of the Panchromatic Image Using the Convolutional Neural Network

Learn to Be Clear and Colorful: An End-to-End Network for Panchromatic Image Enhancement

Hyperspectral Image Super-Resolution Via Convolutional Neural Network

Spatial and Spectral Joint Super-Resolution Using Convolutional Neural Network.

A Super-Resolution Algorithm Based on Hybrid Network for Multi-Channel Remote Sensing Images

Structure–Color Preserving Network for Hyperspectral Image Super-Resolution

Remote Sensing Image Super-Resolution Using Enriched Spatial-Channel Feature Aggregation Networks

HASIC-Net: Hybrid Attentional Convolutional Neural Network with Structure Information Consistency for Spectral Super-Resolution of RGB Images.

Enhanced Context Attention Network for Image Super Resolution

Non-Locally up-Down Convolutional Attention Network for Remote Sensing Image Super-Resolution

RFCNet: Remote Sensing Image Super-Resolution Using Residual Feature Calibration Network

Pixel-Wise Attention Residual Network for Super-Resolution of Optical Remote Sensing Images.

Channel-Wise and Spatial Feature Modulation Network for Single Image Super-Resolution

Remote Sensing Image Super-Resolution Based on Dense Channel Attention Network

Fused Recurrent Network Via Channel Attention for Remote Sensing Satellite Image Super-Resolution