Abstract:Hyperspectral image classification (HSI) is the process of segmenting an image into distinct land cover types by analyzing the rich spectral information of each pixel, with the key lying in feature extraction. Benefiting from the superior ability to exploit long-range dependencies, transformer-based methods have garnered significant attention in the field. However, the limited local sensitivity, high computation burden, influence from heterogeneous spectrum random, and initialization of class token without prior knowledge may restrict the performance of transformer-based methods. To effectively address the aforementioned issues, this study introduces the Dual-Layer Spectral-Spatial Transformer architecture, adept at comprehensively extracting and modeling features. First, to address the issue of limited local sensitivity, we propose a dual-layer transformer architecture, where the inner Pixel-Transformer ensures adequate extraction of local features, and the outer Patch-Transformer is engineered to capture joint spectral-spatial features, thereby strengthening global context modeling. This dual-layer cascading approach not only provides balanced enhancement in feature extraction and modeling, but also alleviates the computational burden associated with self-attention operations. Meanwhile, we have also incorporated a feature selector to mitigate the influence of the heterogeneous spectrum. In addition, the inner Pixel-Transformer enhances feature representation by integrating the spectral vector of the target pixel as a class token, thereby solving the issue of random initialization of the class token without prior knowledge. Experimental results on four public HSI benchmark datasets demonstrate that our model outperforms state-of-the-art methods, with an improvement ranging from 0.86% to a maximum of 3.9%, and has achieved excellent classification results at the boundaries between different land cover types.

FeatureFlow Transformer: Enhancing Feature Fusion and Position Information Modeling for Hyperspectral Image Classification

A Novel Transformer Network with a CNN-Enhanced Cross-Attention Mechanism for Hyperspectral Image Classification

Multiscale 3-D-2-D Mixed CNN and Lightweight Attention-Free Transformer for Hyperspectral and LiDAR Classification

Adaptive Pixel-Level and Superpixel-Level Feature Fusion Transformer for Hyperspectral Image Classification

Expeditious Hyperspectral Image Classification With Inner and Outer Layered Transformer Using Feature Enhancement

Adaptive Learnable Spectral–Spatial Fusion Transformer for Hyperspectral Image Classification

Hyperspectral Image Transformer Classification Networks

Double-branch feature fusion transformer for hyperspectral image classification

Spectral–Spatial Feature Extraction for Hyperspectral Image Classification Using Enhanced Transformer with Large-Kernel Attention

Cascaded Convolution-Based Transformer With Densely Connected Mechanism for Spectral–Spatial Hyperspectral Image Classification

Spectral-Swin Transformer with Spatial Feature Extraction Enhancement for Hyperspectral Image Classification

Multi-layer feature fusion for hyperspectral image classification

A Lightweight Transformer Network for Hyperspectral Image Classification

Convolution Transformer Fusion Splicing Network for Hyperspectral Image Classification

ELS2T: Efficient Lightweight Spectral-Spatial Transformer for Hyperspectral Image Classification

Bridging CNN and Transformer With Cross-Attention Fusion Network for Hyperspectral Image Classification

GTFN: GCN and Transformer Fusion Network With Spatial-Spectral Features for Hyperspectral Image Classification

Fast Hyperspectral Image Classification Combining Transformers and SimAM-Based CNNs

A hybrid convolution transformer for hyperspectral image classification

Classification of hyperspectral and LiDAR data by transformer-based enhancement

A multimodal hyper-fusion transformer for remote sensing image classification