Abstract:Spectral super-resolution (SSR) refers to the hyperspectral image (HSI) recovery from an RGB counterpart. Due to the one-to-many nature of the SSR problem, a single RGB image can be reprojected to many HSIs. The key to tackle this ill-posed problem is to plug into multisource prior information such as the natural spatial context prior of RGB images, deep feature prior, or inherent statistical prior of HSIs so as to effectively alleviate the degree of ill-posedness. However, most current approaches only consider the general and limited priors in their customized convolutional neural networks (CNNs), which leads to the inability to guarantee the confidence and fidelity of reconstructed spectra. In this article, we propose a novel holistic prior-embedded relation network (HPRN) to integrate comprehensive priors to regularize and optimize the solution space of SSR. Basically, the core framework is delicately assembled by several multiresidual relation blocks (MRBs) that fully facilitate the transmission and utilization of the low-frequency content prior of RGBs. Innovatively, the semantic prior of RGB inputs is introduced to mark category attributes, and a semantic-driven spatial relation module (SSRM) is invented to perform the feature aggregation of clustered similar ranges for refining recovered characteristics. In addition, we develop a transformer-based channel relation module (TCRM), which breaks the habit of employing scalars as the descriptors of channelwise relations in the previous deep feature prior and replaces them with certain vectors to make the mapping function more robust and smoother. In order to maintain the mathematical correlation and spectral consistency between hyperspectral bands, the second-order prior constraints (SOPCs) are incorporated into the loss function to guide the HSI reconstruction. Finally, extensive experimental results on four benchmarks demonstrate that our HPRN can reach the state-of-the-art performance for SSR quantitatively and qualitatively. Furthermore, the effectiveness and usefulness of the reconstructed spectra are verified by the classification results on the remote sensing dataset. Codes are available at https://github.com/Deep-imagelab/HPRN.

AdaptiveWeighted Attention Network with Camera Spectral Sensitivity Prior for Spectral Reconstruction from RGB Images

Hybrid 2-D–3-D Deep Residual Attentional Network With Structure Tensor Constraints for Spectral Super-Resolution of RGB Images

Attention Network with Outdoor Illumination Variation Prior for Spectral Reconstruction from RGB Images

Lightweight Multiresolution Feature Fusion Network for Spectral Super-Resolution

Deep Hybrid 2-D–3-D CNN Based on Dual Second-Order Attention with Camera Spectral Sensitivity Prior for Spectral Super-Resolution

Hyperspectral Image Reconstruction from RGB Input Through Highlighting Intrinsic Properties

Learning Hyperspectral Images from RGB Images Via a Coarse-to-fine CNN.

Reconstructing hyperspectral images from rgb inputs based on intrinsic image decomposition

Hyperspectral Image Reconstruction Based on Improved Residual Dense Network

Hyperspectral Image Super-Resolution with 1D-2D Attentional Convolutional Neural Network.

HSCNN+: Advanced CNN-Based Hyperspectral Recovery from RGB Images

Semantic-embedded Unsupervised Spectral Reconstruction from Single RGB Images in the Wild

Hyperspectral Imagery Super-Resolution Based on Self-Calibrated Attention Residual Network.

A Rehabilitation of Pixel-Based Spectral Reconstruction from RGB Images

Hyperspectral Reconstruction Using Auxiliary Rgb Learning From A Snapshot Image

Spectral Image Reconstruction Using Recovered Basis Vector Coefficients

Deep Residual Attention Network For Spectral Image Super-Resolution

HPRN: Holistic Prior-Embedded Relation Network for Spectral Super-Resolution

Residual Spectral–Spatial Attention Network for Hyperspectral Image Classification

Unsupervised spectral reconstruction from RGB images under two lighting conditions

Spectral Response Function-Guided Deep Optimization-Driven Network for Spectral Super-Resolution