Abstract:Hyperspectral (HS) image classification has become an important research area. Although previous work on HS image classification has achieved impressive results, finding a proper balance between extracting spatial–spectral information and capturing band similarities remains a challenge. To address this problem, we designed some efficient modules and constructed hybrid covariance attention and cross-layer fusion transformer network (CACFTNet), which efficiently models the extraction of spatial–spectral information and band similarity. First, our approach combines local and global perspectives to process the desired feature information. We introduce the dual-branch feature processing (DBFP) module, which can model the desired spatial–spectral information and band similarity. Second, we design the dual-branch feature fusion (DBFF) module, which combines a convolutional neural network (CNN) and a transformer to catch spatial–spectral information at different scales and fuse them effectively. To further process the features, we construct the unparameterized covariance attention (UPCA) module, which utilizes the covariance matrix to capture the correlation between different spectral bands. This allows the network to concentrate on bands that are more useful for classification tasks. In addition, we designed a hybrid activation function (HAF) that maps the channel values to specific ranges and emphasizes the extent to which the correlation varies between different bands. Finally, in order to incorporate important information from different layers, we propose the cross-layer adaptive attention fusion (CAAF) module, which fully fuses information between layers and enriches the overall information representation. We evaluate our proposed model on three well-known public datasets and demonstrate its superiority over existing approaches.

Bridging CNN and Transformer With Cross-Attention Fusion Network for Hyperspectral Image Classification

A Novel Transformer Network with a CNN-Enhanced Cross-Attention Mechanism for Hyperspectral Image Classification

Multiscale 3-D-2-D Mixed CNN and Lightweight Attention-Free Transformer for Hyperspectral and LiDAR Classification

Convolution-Transformer Adaptive Fusion Network for Hyperspectral Image Classification

CACFTNet:A Hybrid Cov-Attention and Cross-Layer Fusion Transformer Network for Hyperspectral Image Classification

CACFTNet: A Hybrid Cov-Attention and Cross-Layer Fusion Transformer Network for Hyperspectral Image Classification

Hyperspectral Image Classification Using Hierarchical Spatial-Spectral Transformer

Convolution Transformer Fusion Splicing Network for Hyperspectral Image Classification

CNN and Transformer Hybrid Network for Hyperspectral Image Classification

Multiscale and Cross-Level Attention Learning for Hyperspectral Image Classification

Double-branch feature fusion transformer for hyperspectral image classification

Hyperspectral Image Classification Based on Interactive Transformer and CNN With Multilevel Feature Fusion Network

CNN and Transformer interaction network for hyperspectral image classification

A Cross-Attention-Based Multi-information Fusion Transformer for Hyperspectral Image Classification

Fusion of Dilated Convolution in CNN and Transformer Networks for Hyperspectral Image Classification

A Joint Convolutional Cross ViT Network for Hyperspectral and Light Detection and Ranging Fusion Classification

Multi-layer feature fusion for hyperspectral image classification

DCTN: Dual-Branch Convolutional Transformer Network With Efficient Interactive Self-Attention for Hyperspectral Image Classification

End-to-End Convolutional Network and Spectral-Spatial Transformer Architecture for Hyperspectral Image Classification

Adaptive Pixel-Level and Superpixel-Level Feature Fusion Transformer for Hyperspectral Image Classification

Fast Hyperspectral Image Classification Combining Transformers and SimAM-Based CNNs