Abstract:RGB image spectral super-resolution (SSR) is a challenging task due to its serious ill-posedness, which aims at recovering a hyperspectral image (HSI) from a corresponding RGB image. In this article, we propose a novel hybrid 2-D–3-D deep residual attentional network (HDRAN) with structure tensor constraints, which can take fully advantage of the spatial–spectral context information in the reconstruction progress. Previous works improve the SSR performance only through stacking more layers to catch local spatial correlation neglecting the differences and interdependences among features, especially band features; different from them, our novel method focuses on the context information utilization. First, the proposed HDRAN consists of a 2D-RAN following by a 3D-RAN, where the 2D-RAN mainly focuses on extracting abundant spatial features, whereas the 3D-RAN mainly simulates the interband correlations. Then, we introduce 2-D channel attention and 3-D band attention mechanisms into the 2D-RAN and 3D-RAN, respectively, to adaptively recalibrate channelwise and bandwise feature responses for enhancing context features. Besides, since structure tensor represents structure and spatial information, we apply structure tensor constraint to further reconstruct more accurate high-frequency details during the training process. Experimental results demonstrate that our proposed method achieves the state-of-the-art performance in terms of mean relative absolute error (MRAE) and root mean square error (RMSE) on both the "clean" and "real world" tracks in the NTIRE 2018 Spectral Reconstruction Challenge. As for competitive ranking metric MRAE, our method separately achieves a 16.06% and 2.90% relative reduction on two tracks over the first place. Furthermore, we investigate HDRAN on the other two HSI benchmarks noted as t-e CAVE and Harvard data sets, also demonstrating better results than state-of-the-art methods.

Neighbor Spectra Maintenance and Context Affinity Enhancement for Single Hyperspectral Image Super-Resolution

Asymmetric Dual-Direction Quasi-Recursive Network for Single Hyperspectral Image Super-Resolution

Exploring the Spectral Prior for Hyperspectral Image Super-Resolution

A Spectral Grouping and Attention-Driven Residual Dense Network for Hyperspectral Image Super-Resolution

TC-HISRNet: Hyperspectral Image Super-Resolution Network Based on Contextual Band Joint Transformer and CNN

Hyperspectral Image Super-Resolution via Dual-domain Network Based on Hybrid Convolution

A Diffusion Model-Assisted Multiscale Spectral Attention Network for Hyperspectral Image Super-Resolution

Attention-Enhanced Generative Adversarial Network for Hyperspectral Imagery Spatial Super-Resolution

Spatial-Spectral Middle Cross-Attention Fusion Network for Hyperspectral Image Superresolution

Hyperspectral Image Super-resolution via Deep Spatio-spectral Convolutional Neural Networks

Spectral Domain Strategies for Hyperspectral Super-Resolution: Transfer Learning and Channel Enhance Network

Hyperspectral Image Super-Resolution via Deep Spatiospectral Attention Convolutional Neural Networks

Integration of Spatial–Spectral Information for Resolution Enhancement in Hyperspectral Images

An Efficient Residual Learning Neural Network for Hyperspectral Image Superresolution

Difference Curvature Multidimensional Network for Hyperspectral Image Super-Resolution

Hybrid 2-D–3-D Deep Residual Attentional Network With Structure Tensor Constraints for Spectral Super-Resolution of RGB Images

Hyperspectral Image Super-Resolution With ConvLSTM Skip-Connections

Hyper-Spectral Image Super-Resolution Using Non-Negative Spectral Representation with Data-Guided Sparsity

Spectral-Enhanced Sparse Transformer Network for Hyperspectral Super-Resolution Reconstruction

Hyperspectral image super-resolution via a multi-stage scheme without employing spatial degradation

Hyperspectral Image Super-Resolution Via Recurrent Feedback Embedding and Spatial–Spectral Consistency Regularization