Abstract:Due to the limitations of imaging systems, the swath width of available hyperspectral images (HSIs) is relatively narrow, making it difficult to meet the demands of various applications. Therefore, an appealing idea is to expand the width of HSIs by using widely covered multispectral images (MSIs), called spectral super-resolution (SSR) of MSIs. According to the radiation transmission process of the imaging system, the spectral mixing characteristics of ground objects can be described by the linear spectral mixing model (LSMM). Inspired by the linear mixed part and nonlinear residual part of the LSMM, we propose a double-branch SSR network. To generate wide HSIs, a spectral mixing branch is designed to extract abundances from wide MSIs and adaptively learn hyperspectral endmembers from narrow HSIs. Furthermore, considering the nonlinear factors in the imaging system and atmospheric transmission, a nonlinear residual branch is built to complement the spectral and spatial details. Finally, the SSR result can be obtained with the fusion of linear and nonlinear features. To make the network structure achieve corresponding physical significance, we constrain the network through joint loss functions at different stages. In addition to two simulated datasets with limited coverage, our model is also evaluated on a real MSI–HSI dataset in a larger area. Extensive experiments show the superiority of the proposed model compared with state-of-the-art baselines. Moreover, we visualized the internal results of our network and conducted ablation experiments on a single branch to further demonstrate its effectiveness. In the end, the influence of network hyperparameters, including endmembers and loss function weight coefficient, is discussed and analyzed in detail.

Spectral Reconstruction for Paired Images Based on Semi-supervised Deep Learning

Spectral Reconstruction Network From Multispectral Images to Hyperspectral Images: A Multitemporal Case

Hyperspectral Image Super Resolution Reconstruction with a Joint Spectral-Spatial Sub-Pixel Mapping Model.

Hyperspectral Imagery Spatial Super-Resolution Using Generative Adversarial Network.

Multi-sensor Multispectral Reconstruction Framework Based on Projection and Reconstruction

Hyperspectral Image Reconstruction from RGB Input Through Highlighting Intrinsic Properties

Reconstructing hyperspectral images from rgb inputs based on intrinsic image decomposition

A hyperspectral image denoising method based on land cover spectral autocorrelation

A SPATIAL CONSTRAINT AND DEEP LEARNING BASED HYPERSPECTRAL IMAGE SUPER-RESOLUTION METHOD

Unsupervised spectral reconstruction from RGB images under two lighting conditions

Semantic-embedded Unsupervised Spectral Reconstruction from Single RGB Images in the Wild

Spectral Super-Resolution for Multispectral Image Based on Spectral Improvement Strategy and Spatial Preservation Strategy.

Spectral Response Function-Guided Deep Optimization-Driven Network for Spectral Super-Resolution

Spectral super-resolution meets deep learning: Achievements and challenges

Remote Sensing Hyperspectral Image Super-Resolution via Multidomain Spatial Information and Multiscale Spectral Information Fusion

Spectral-Spatial Latent Reconstruction for Open-Set Hyperspectral Image Classification

Spectral Mixing Theory-Based Double-Branch Network for Spectral Super-Resolution

Superpixel-Based Hsi Classification Via Semi-Supervised K-Svd And Multi-Scale Sparse Representation

Multi-Task Interaction Learning for Spatiospectral Image Super-Resolution.

Hyperspectral Image Super-Resolution by Spectral Difference Learning and Spatial Error Correction

Spectral Reconstruction From Satellite Multispectral Imagery Using Convolution and Transformer Joint Network