Abstract:RGB image spectral super-resolution (SSR) is a challenging task due to its serious ill-posedness, which aims at recovering a hyperspectral image (HSI) from a corresponding RGB image. In this article, we propose a novel hybrid 2-D–3-D deep residual attentional network (HDRAN) with structure tensor constraints, which can take fully advantage of the spatial–spectral context information in the reconstruction progress. Previous works improve the SSR performance only through stacking more layers to catch local spatial correlation neglecting the differences and interdependences among features, especially band features; different from them, our novel method focuses on the context information utilization. First, the proposed HDRAN consists of a 2D-RAN following by a 3D-RAN, where the 2D-RAN mainly focuses on extracting abundant spatial features, whereas the 3D-RAN mainly simulates the interband correlations. Then, we introduce 2-D channel attention and 3-D band attention mechanisms into the 2D-RAN and 3D-RAN, respectively, to adaptively recalibrate channelwise and bandwise feature responses for enhancing context features. Besides, since structure tensor represents structure and spatial information, we apply structure tensor constraint to further reconstruct more accurate high-frequency details during the training process. Experimental results demonstrate that our proposed method achieves the state-of-the-art performance in terms of mean relative absolute error (MRAE) and root mean square error (RMSE) on both the "clean" and "real world" tracks in the NTIRE 2018 Spectral Reconstruction Challenge. As for competitive ranking metric MRAE, our method separately achieves a 16.06% and 2.90% relative reduction on two tracks over the first place. Furthermore, we investigate HDRAN on the other two HSI benchmarks noted as t-e CAVE and Harvard data sets, also demonstrating better results than state-of-the-art methods.

RAW2HSI: Learning-Based Hyperspectral Image Reconstruction from Low-Resolution Noisy Raw-RGB

Compressive Single-Pixel Hyperspectral Imaging Using RGB Sensors

Metadata-Based RAW Reconstruction Via Implicit Neural Functions.

Hyperspectral Reconstruction From RGB Images via Physically Guided Graph Deep Prior Learning

Semantic-embedded Unsupervised Spectral Reconstruction from Single RGB Images in the Wild

Hyperspectral Image Super-Resolution via Dual-domain Network Based on Hybrid Convolution

Asymmetric Dual-Direction Quasi-Recursive Network for Single Hyperspectral Image Super-Resolution

HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging

Hyperspectral Compressive Snapshot Reconstruction via Coupled Low-Rank Subspace Representation and Self-Supervised Deep Network

Deep Amended Gradient Descent for Efficient Spectral Reconstruction from Single RGB Images

Hyperspectral Image Super-Resolution via Deep Spatiospectral Attention Convolutional Neural Networks

Deep Self-Supervised Hyperspectral Image Reconstruction

Hyperspectral Image Super-Resolution with a Mosaic RGB Image

Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery

Blind Super-Resolution of Single Remotely Sensed Hyperspectral Image

Remote Sensing Hyperspectral Image Super-Resolution via Multidomain Spatial Information and Multiscale Spectral Information Fusion

Tensor-Based Sparse Representation for Hyperspectral Image Reconstruction Using RGB Inputs

Hyperspectral image dynamic range reconstruction using deep neural network-based denoising methods

Unsupervised Spatial-spectral Network Learning for Hyperspectral Compressive Snapshot Reconstruction

Hybrid 2-D–3-D Deep Residual Attentional Network With Structure Tensor Constraints for Spectral Super-Resolution of RGB Images

Grayscale to Hyperspectral at Any Resolution Using a Phase-Only Lens