A dual-branch siamese spatial-spectral transformer attention network for Hyperspectral Image Change Detection

Yiyan Zhang,Tingting Wang,Chenkai Zhang,Shufang Xu,Hongmin Gao,Chenming Li
DOI: https://doi.org/10.1016/j.eswa.2023.122125
IF: 8.5
2023-10-27
Expert Systems with Applications
Abstract:The convolutional neural networks have recently gained widespread attention in Hyperspectral Image Change Detection (HSI-CD) due to their outstanding feature extraction ability. However, limited by the inherent network backbones, the convolutional neural networks (CNNs) fail to mine the sequence attributes and model the intricate relationships of spectral signatures. In contrast, transformers are proficient at learning sequence information owing to the powerful self-attention mechanisms. The two backbone structures exhibit complementary spatial and spectral feature extraction strengths, respectively. Inspired by this, we propose a dual-branch siamese spatial–spectral transformer attention network (DBS 3 TAN) for HSI-CD. The main idea is to fully exploit the advantages of CNNs and transformers for spatial and spectral feature extraction. More importantly, we devise the two key modules, i.e., the spatial attention module and the spatial–spectral transformer module. The former utilizes depthwise separable convolutions and attention mechanisms to emphasize the features of dual-temporal HSIs from the spatial perspective. The latter focuses on the sequence attributes of spectral signatures and mines the spatial characteristics from adjacent pixels. We employ the weighted contrastive loss function to separate the changed and unchanged pixels more reliably and set the random weight factors to balance the contributions of the two branches. Finally, the threshold values judgment is used to obtain the ultimate detection maps. We conduct extensive experiments to evaluate the DBS 3 TAN on three HSI datasets, demonstrating its superior performances than compared methods qualitatively and quantitatively. The source code will be available at https://github.com/zhangyiyan001/DBS3TAN .
computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science
What problem does this paper attempt to address?