Abstract:Many applications have benefited from the images with both high spatial and spectral resolution, such as mineralogy and surveillance. However, it is difficult to acquire such images due to the limitation of sensor technologies. Recently, super-resolution (SR) techniques have been proposed to improve the spatial or spectral resolution of images, e.g., improving the spatial resolution of hyperspectral images (HSIs) or improving spectral resolution of color images (reconstructing HSIs from RGB inputs). However, none of the researches attempted to improve both spatial and spectral resolution together. In this article, these two types of resolution are jointly improved using convolutional neural network (CNN). Specifically, two kinds of CNN-based SR are conducted, including a simultaneous spatial-spectral joint SR (SimSSJSR) that conducts SR in spectral and spatial domain simultaneously and a separated spatial-spectral joint SR (SepSSJSR) that considers spectral and spatial SR sequentially. In the proposed SimSSJSR, a full 3-D CNN is constructed to learn an end-to-end mapping between a low spatial-resolution mulitspectral image (LR-MSI) and the corresponding high spatial-resolution HSI (HR-HSI). In the proposed SepSSJSR, a spatial SR network and a spectral SR network are designed separately, and thus two different frameworks are proposed for SepSSJSR, namely SepSSJSR1 and SepSSJSR2, according to the order that spatial SR and spectral SR are applied. Furthermore, the least absolute deviation, instead of mean square error (MSE) in traditional SR networks, is chosen as the loss function for the proposed networks. Experimental results over simulated images from different sensors demonstrated that the proposed SepSSJSR1 is most effective to improve spatial and spectral resolution of MSIs sequentially by conducting spatial SR prior to spectral SR. In addition, validation on real Landsat images also indicates that the proposed SSJSR techniques can make full use of available MSIs for high-resolution-based analysis or applications.

Simultaneous Super-Resolution and Segmentation for Remote Sensing Images

Parallel-Connected Residual Channel Attention Network for Remote Sensing Image Super-Resolution

Superpixel-Based Long-Range Dependent Network for High-Resolution Remote-Sensing Image Classification

Semantic segmentation of remote sensing images combined with attention mechanism and feature enhancement U-Net

Collaborative Network for Super-Resolution and Semantic Segmentation of Remote Sensing Images

ISDNet: Integrating Shallow and Deep Networks for Efficient Ultra-high Resolution Segmentation

A Dual Network for Super-Resolution and Semantic Segmentation of Sentinel-2 Imagery

Spatial and Spectral Joint Super-Resolution Using Convolutional Neural Network.

Super-Resolution for Remote Sensing Images Via Local–Global Combined Network

SEG-ESRGAN: A Multi-Task Network for Super-Resolution and Semantic Segmentation of Remote Sensing Images

An Object-Aware Network Embedding Deep Superpixel for Semantic Segmentation of Remote Sensing Images

RSI-Net: Two-Stream Deep Neural Network for Remote Sensing Images-Based Semantic Segmentation

Hybrid-Scale Self-Similarity Exploitation for Remote Sensing Image Super-Resolution

High-Resolution Remote Sensing Image Segmentation Algorithm Based on Improved Feature Extraction and Hybrid Attention Mechanism

Multi-Stage Fusion and Multi-Source Attention Network for Multi-Modal Remote Sensing Image Segmentation.

SRDA-Net: Super-Resolution Domain Adaptation Networks for Semantic Segmentation.

DHRNet: A Dual-Branch Hybrid Reinforcement Network for Semantic Segmentation of Remote Sensing Images

Multi-scale attention fusion network for semantic segmentation of remote sensing images

Multi-scale Spatial Aggregation Network for Remote Sensing Image Segmentation

An Attention-Fused Network for Semantic Segmentation of Very-High-Resolution Remote Sensing Imagery

Remote Sensing Image Super-Resolution Via Dual-Resolution Network Based on Connected Attention Mechanism