Abstract:Many applications have benefited from the images with both high spatial and spectral resolution, such as mineralogy and surveillance. However, it is difficult to acquire such images due to the limitation of sensor technologies. Recently, super-resolution (SR) techniques have been proposed to improve the spatial or spectral resolution of images, e.g., improving the spatial resolution of hyperspectral images (HSIs) or improving spectral resolution of color images (reconstructing HSIs from RGB inputs). However, none of the researches attempted to improve both spatial and spectral resolution together. In this article, these two types of resolution are jointly improved using convolutional neural network (CNN). Specifically, two kinds of CNN-based SR are conducted, including a simultaneous spatial-spectral joint SR (SimSSJSR) that conducts SR in spectral and spatial domain simultaneously and a separated spatial-spectral joint SR (SepSSJSR) that considers spectral and spatial SR sequentially. In the proposed SimSSJSR, a full 3-D CNN is constructed to learn an end-to-end mapping between a low spatial-resolution mulitspectral image (LR-MSI) and the corresponding high spatial-resolution HSI (HR-HSI). In the proposed SepSSJSR, a spatial SR network and a spectral SR network are designed separately, and thus two different frameworks are proposed for SepSSJSR, namely SepSSJSR1 and SepSSJSR2, according to the order that spatial SR and spectral SR are applied. Furthermore, the least absolute deviation, instead of mean square error (MSE) in traditional SR networks, is chosen as the loss function for the proposed networks. Experimental results over simulated images from different sensors demonstrated that the proposed SepSSJSR1 is most effective to improve spatial and spectral resolution of MSIs sequentially by conducting spatial SR prior to spectral SR. In addition, validation on real Landsat images also indicates that the proposed SSJSR techniques can make full use of available MSIs for high-resolution-based analysis or applications.

Alternative deep learning method for fast spatial-frequency shift imaging microscopy.

Untrained Neural Network Enabling Fast and Universal Structured-Illumination Microscopy

Chip-compatible wide-field 3D nanoscopy through tunable spatial frequency shift effect

Universal and High-Fidelity Resolution Extending for Fluorescence Microscopy Using a Single-Training Physics-Informed Sparse Neural Network

Wavefront Reconstruction Based on Deep Transfer Learning for Microscopy.

Spatial and Spectral Joint Super-Resolution Using Convolutional Neural Network.

Wide-Field, High-Resolution Reconstruction in Computational Multi-Aperture Miniscope Using a Fourier Neural Network

Ultracompact Deep Neural Network for Ultrafast Optical Property Extraction in Spatial Frequency Domain Imaging (SFDI)

Deep Learning Model for Ultrafast Multifrequency Optical Property Extractions for Spatial Frequency Domain Imaging.

Fast High-Resolution Imaging Combining Deep Learning and Single-Pixel Imaging

Fast light-field 3D microscopy with out-of-distribution detection and adaptation through Conditional Normalizing Flows

Artificial confocal microscopy for deep label-free imaging

De-scattering Deep Neural Network Enables Fast Imaging of Spines Through Scattering Media by Temporal Focusing Microscopy

Fourier single pixel imaging reconstruction method based on the U-net and attention mechanism at a low sampling rate

High-axial-resolution Single-Molecule Localization under Dense Excitation with a Multi-Channel Deep U-Net.

Fast Confocal Microscopy Imaging Based on Deep Learning

Real-Time Resolution Enhancement of Confocal Laser Scanning Microscopy via Deep Learning

TAG-SPARK: Empowering High-Speed Volumetric Imaging With Deep Learning and Spatial Redundancy

Fast and Lightweight Network for Single Frame Structured Illumination Microscopy Super-Resolution

Double‐flow convolutional neural network for rapid large field of view Fourier ptychographic reconstruction

High-quality Label-Free Prediction of Fluorescence Images Through Stimulated Raman Scattering Imaging and Recurrent Deep Neural Network.