Abstract:Hyperspectral image (HSI) super-resolution is a practical and challenging task as it requires the reconstruction of a large number of spectral bands. Achieving excellent reconstruction results can greatly benefit subsequent downstream tasks. The current mainstream hyperspectral super-resolution methods mainly utilize 3D convolutional neural networks (3D CNN) for design. However, the commonly used small kernel size in 3D CNN limits the model’s receptive field, preventing it from considering a wider range of contextual information. Though the receptive field could be expanded by enlarging the kernel size, it results in a dramatic increase in model parameters. Furthermore, the popular vision transformers designed for natural images are not suitable for processing HSI. This is because HSI exhibits sparsity in the spatial domain, which can lead to significant computational resource waste when using self-attention. In this paper, we design a hybrid architecture called HyFormer, which combines the strengths of CNN and transformer for hyperspectral super-resolution. The transformer branch enables intra-spectra interaction to capture fine-grained contextual details at each specific wavelength. Meanwhile, the CNN branch facilitates efficient inter-spectra feature extraction among different wavelengths while maintaining a large receptive field. Specifically, in the transformer branch, we propose a novel Grouping-Aggregation transformer (GAT), comprising grouping self-attention (GSA) and aggregation self-attention (ASA). The GSA is employed to extract diverse fine-grained features of targets, while the ASA facilitates interaction among heterogeneous textures allocated to different channels. In the CNN branch, we propose a Wide-Spanning Separable 3D Attention (WSSA) to enlarge the receptive field while keeping a low parameter number. Building upon WSSA, we construct a wide-spanning CNN module to efficiently extract inter-spectra features. Extensive experiments demonstrate the superior performance of our HyFormer.

Cross-Scope Spatial-Spectral Information Aggregation for Hyperspectral Image Super-Resolution

Cross-Scope Spatial-Spectral Information Aggregation for Hyperspectral Image Super-Resolution

Cognitively-Inspired Multi-Scale Spectral-Spatial Transformer for Hyperspectral Image Super-Resolution

A novel spatial and spectral transformer network for hyperspectral image super-resolution

Spectral Superresolution Using Transformer with Convolutional Spectral Self-Attention

A spectral and spatial transformer for hyperspectral remote sensing image super-resolution

Hyperspectral Image Super-Resolution Via Convolutional Neural Network

MSDformer: Multiscale Deformable Transformer for Hyperspectral Image Super-Resolution

Hyperspectral Image Super-Resolution with 1D-2D Attentional Convolutional Neural Network.

Hyperspectral Image Spatial Super-Resolution Via 3d Full Convolutional Neural Network

Remote Sensing Image Super-Resolution Via Cross-Scale Hierarchical Transformer

Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction

HyFormer: Hybrid Grouping-Aggregation Transformer and Wide-Spanning CNN for Hyperspectral Image Super-Resolution

Spectral-Spatial Transformer for Hyperspectral Image Sharpening

Spatial-Spectral Middle Cross-Attention Fusion Network for Hyperspectral Image Superresolution

Remote Sensing Hyperspectral Image Super-Resolution via Multidomain Spatial Information and Multiscale Spectral Information Fusion

Spectral Super-Resolution via Model-Guided Cross-Fusion Network

Learning Exhaustive Correlation for Spectral Super-Resolution: Where Spatial-Spectral Attention Meets Linear Dependence

HCT: a hybrid CNN and transformer network for hyperspectral image super-resolution

Hyperspectral Image Super-Resolution via Deep Spatiospectral Attention Convolutional Neural Networks

Hyperspectral Image Super-Resolution Network Based on Cross-Scale Nonlocal Attention.