Hyperspectral Image Denoising via Spatial-Spectral Recurrent Transformer

Guanyiman Fu,Fengchao Xiong,Jianfeng Lu,Jun Zhou,Jiantao Zhou,Yuntao Qian

DOI: https://doi.org/10.1109/TGRS.2024.3374953

2024-01-09

Abstract:Hyperspectral images (HSIs) often suffer from noise arising from both intra-imaging mechanisms and environmental factors. Leveraging domain knowledge specific to HSIs, such as global spectral correlation (GSC) and non-local spatial self-similarity (NSS), is crucial for effective denoising. Existing methods tend to independently utilize each of these knowledge components with multiple blocks, overlooking the inherent 3D nature of HSIs where domain knowledge is strongly interlinked, resulting in suboptimal performance. To address this challenge, this paper introduces a spatial-spectral recurrent transformer U-Net (SSRT-UNet) for HSI denoising. The proposed SSRT-UNet integrates NSS and GSC properties within a single SSRT block. This block consists of a spatial branch and a spectral branch. The spectral branch employs a combination of transformer and recurrent neural network to perform recurrent computations across bands, allowing for GSC exploitation beyond a fixed number of bands. Concurrently, the spatial branch encodes NSS for each band by sharing keys and values with the spectral branch under the guidance of GSC. This interaction between the two branches enables the joint utilization of NSS and GSC, avoiding their independent treatment. Experimental results demonstrate that our method outperforms several alternative approaches. The source code will be available at

Image and Video Processing,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem this paper attempts to address is the noise introduced in hyperspectral images (HSI) during acquisition due to imaging mechanisms and environmental factors. Hyperspectral images typically contain a large amount of noise, which affects the quality and practical application value of the images. Existing denoising methods often independently utilize domain knowledge such as non-local spatial self-similarity (NSS) and global spectral correlation (GSC), while ignoring the inherent three-dimensional characteristics of HSI, leading to suboptimal performance.

To tackle this challenge, the paper proposes a U-Net architecture based on a spatial-spectral recursive transformer (SSRT-UNet) for hyperspectral image denoising. SSRT-UNet integrates NSS and GSC properties within a single SSRT block, achieving joint utilization of these two attributes. Specifically, the SSRT block contains a spatial branch and a spectral branch. The spectral branch combines transformers and recurrent neural networks to leverage GSC through recursive computation across bands, surpassing the limitation of a fixed number of bands. Meanwhile, the spatial branch encodes the NSS of each band under the guidance of GSC, interacting with the spectral branch by sharing keys and values, thereby achieving joint utilization of NSS and GSC.

Experimental results show that this method outperforms existing methods on multiple benchmark datasets, effectively improving the denoising performance of hyperspectral images.