Abstract:Super-resolution is vital for the quality improvement of hyperspectral images (HSIs) under the spatial and spectral resolution trade-off. However, deep learning HSI super-resolution approaches typically adopt the "one model and one scale" scheme that is inefficient in training and storing. This is difficult in maximizing orbit equipment performance and aligning multiple spatial resolution data in remote sensing. Therefore, this article intends to address HSI arbitrary-scale super-resolution, enabling the scaling of HSIs to arbitrary sizes using a single model. To do this end, we treat HSI arbitrary-scale super-resolution as a retrieval problem. It conceptualizes the HSI as a dictionary of pixelwise tokens with spatial-spectral features, position information, and scale information. Its objective is to employ a set of initialized tokens related to the high-resolution (HR) HSI as queries to retrieve matched spectral features from low-resolution (LR) one, which is so-called token-based query-to-spectrum. Since these query tokens can be constructed flexibly (e.g., through random initialization), we can generate a desired number of them to reconstruct our HR HSI, thus achieving arbitrary-scale super-resolution. This process considers not only position information but also spectral features so that it can decrease spectral distortion. With the above idea, we developed an HSI arbitrary-scale super-resolution method, dubbed as spectral-query transformer (SQformer). Specifically, it begins by converting the LR HSI into a dictionary of LR tokens and then constructs a desired number of HR tokens. To enable flexible token construction, we design an implicit spectral token (particularly a learnable vector) and replicate it times to form the HR tokens. Next, the HR and LR tokens are passed into a transformer decoder to find the most matched spectral response for the former by soft-weighting the LR tokens. Finally, the HR tokens are spatially rearranged in order, forming an HR HSI. Extensive experiments have demonstrated its effectiveness on remote sensing data. The code will be released at: https://github.com/ShuGuoJ/SQformer.git.

Cognitively-Inspired Multi-Scale Spectral-Spatial Transformer for Hyperspectral Image Super-Resolution

Hyperspectral Image Super-Resolution via Deep Spatiospectral Attention Convolutional Neural Networks

Remote Sensing Hyperspectral Image Super-Resolution via Multidomain Spatial Information and Multiscale Spectral Information Fusion

MIMO-SST: Multi-Input Multi-Output Spatial-Spectral Transformer for Hyperspectral and Multispectral Image Fusion

Spectral Superresolution Using Transformer with Convolutional Spectral Self-Attention

A Hybrid Spectral Attention-Enabled Multiscale Spatial–Spectral Learning Network for Hyperspectral Image Superresolution

Cross-Scope Spatial-Spectral Information Aggregation for Hyperspectral Image Super-Resolution

Hyperspectral Image Super-resolution via Deep Spatio-spectral Convolutional Neural Networks

A spectral and spatial transformer for hyperspectral remote sensing image super-resolution

Spectral Super-Resolution via Model-Guided Cross-Fusion Network

Multi-Task Interaction Learning for Spatiospectral Image Super-Resolution.

SQformer: Spectral-Query Transformer for Hyperspectral Image Arbitrary-Scale Super-Resolution

HMFT: Hyperspectral and Multispectral Image Fusion Super-Resolution Method Based on Efficient Transformer and Spatial-Spectral Attention Mechanism

Hyperspectral Image Superresolution via Deep Structure and Texture Interfusion

Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction

Multi-Scale Super Token Transformer for Hyperspectral Image Classification

A novel spatial and spectral transformer network for hyperspectral image super-resolution

Supervised-unsupervised combined transformer for spectral compressive imaging reconstruction

A Diffusion Model-Assisted Multiscale Spectral Attention Network for Hyperspectral Image Super-Resolution

Spectral-Enhanced Sparse Transformer Network for Hyperspectral Super-Resolution Reconstruction