Abstract:Hyperspectral image classification (HSI) is the process of segmenting an image into distinct land cover types by analyzing the rich spectral information of each pixel, with the key lying in feature extraction. Benefiting from the superior ability to exploit long-range dependencies, transformer-based methods have garnered significant attention in the field. However, the limited local sensitivity, high computation burden, influence from heterogeneous spectrum random, and initialization of class token without prior knowledge may restrict the performance of transformer-based methods. To effectively address the aforementioned issues, this study introduces the Dual-Layer Spectral-Spatial Transformer architecture, adept at comprehensively extracting and modeling features. First, to address the issue of limited local sensitivity, we propose a dual-layer transformer architecture, where the inner Pixel-Transformer ensures adequate extraction of local features, and the outer Patch-Transformer is engineered to capture joint spectral-spatial features, thereby strengthening global context modeling. This dual-layer cascading approach not only provides balanced enhancement in feature extraction and modeling, but also alleviates the computational burden associated with self-attention operations. Meanwhile, we have also incorporated a feature selector to mitigate the influence of the heterogeneous spectrum. In addition, the inner Pixel-Transformer enhances feature representation by integrating the spectral vector of the target pixel as a class token, thereby solving the issue of random initialization of the class token without prior knowledge. Experimental results on four public HSI benchmark datasets demonstrate that our model outperforms state-of-the-art methods, with an improvement ranging from 0.86% to a maximum of 3.9%, and has achieved excellent classification results at the boundaries between different land cover types.

Dual Branch Masked Transformer for Hyperspectral Image Classification

A Novel Transformer Network with a CNN-Enhanced Cross-Attention Mechanism for Hyperspectral Image Classification

Multiscale 3-D-2-D Mixed CNN and Lightweight Attention-Free Transformer for Hyperspectral and LiDAR Classification

Expeditious Hyperspectral Image Classification With Inner and Outer Layered Transformer Using Feature Enhancement

A Center-Masked Transformer for Hyperspectral Image Classification

MSMT-LCL: Multiscale Spatial-Spectral Masked Transformer With Local Contrastive Learning for Hyperspectral Image Classification

Hyperspectral Image Classification Using Group-Aware Hierarchical Transformer

Hyperspectral Image Classification Based on Multibranch Attention Transformer Networks

A Dual-Branch Multiscale Transformer Network for Hyperspectral Image Classification

Multi-scale Spectral-Spatial Dual-Transformer Network for Hyperspectral Image Classification

A Multi-Range Spectral-Spatial Transformer for Hyperspectral Image Classification

Cross-Domain Hyperspectral Image Classification Based on Transformer

Dual-Branch Adaptive Convolutional Transformer for Hyperspectral Image Classification

Joint Classification of Hyperspectral Images and LiDAR Data Based on Dual-Branch Transformer

D2S2BoT: Dual-Dimension Spectral-Spatial Bottleneck Transformer for Hyperspectral Image Classification

Hierarchical Feature Fusion of Transformer with Patch Dilating for Remote Sensing Scene Classification

Tripartite‐structure transformer for hyperspectral image classification

S^2-Transformer for Mask-Aware Hyperspectral Image Reconstruction

Spectral–Spatial Feature Extraction for Hyperspectral Image Classification Using Enhanced Transformer with Large-Kernel Attention

Hybrid Multi-Scale Spatial-Spectral Transformer for Hyperspectral Image Classification

Selective Transformer for Hyperspectral Image Classification