Abstract:Hyperspectral (HS) image classification has become an important research area. Although previous work on HS image classification has achieved impressive results, finding a proper balance between extracting spatial–spectral information and capturing band similarities remains a challenge. To address this problem, we designed some efficient modules and constructed hybrid covariance attention and cross-layer fusion transformer network (CACFTNet), which efficiently models the extraction of spatial–spectral information and band similarity. First, our approach combines local and global perspectives to process the desired feature information. We introduce the dual-branch feature processing (DBFP) module, which can model the desired spatial–spectral information and band similarity. Second, we design the dual-branch feature fusion (DBFF) module, which combines a convolutional neural network (CNN) and a transformer to catch spatial–spectral information at different scales and fuse them effectively. To further process the features, we construct the unparameterized covariance attention (UPCA) module, which utilizes the covariance matrix to capture the correlation between different spectral bands. This allows the network to concentrate on bands that are more useful for classification tasks. In addition, we designed a hybrid activation function (HAF) that maps the channel values to specific ranges and emphasizes the extent to which the correlation varies between different bands. Finally, in order to incorporate important information from different layers, we propose the cross-layer adaptive attention fusion (CAAF) module, which fully fuses information between layers and enriches the overall information representation. We evaluate our proposed model on three well-known public datasets and demonstrate its superiority over existing approaches.

Texture-Aware Causal Feature Extraction Network for Multimodal Remote Sensing Data Classification

Dynamic Convolution Covariance Network Using Multi-Scale Feature Fusion for Remote Sensing Scene Image Classification

Ebbinghaus-Curve Guided Low-Rank Component-Induced Attention for Multisource Remote Sensing Classification

Remote Sensing Image Classification Based on Canny Operator Enhanced Edge Features

Spectral–Spatial Enhancement and Causal Constraint for Hyperspectral Image Cross-Scene Classification

Intra- and Intersource Interactive Representation Learning Network for Remote Sensing Images Classification

Dual-Branch Feature Fusion Network Based Cross-Modal Enhanced CNN and Transformer for Hyperspectral and LiDAR Classification

TCPSNet: Transformer and Cross-Pseudo-Siamese Learning Network for Classification of Multi-Source Remote Sensing Images

CACFTNet: A Hybrid Cov-Attention and Cross-Layer Fusion Transformer Network for Hyperspectral Image Classification

CMR-net: A cross modality reconstruction network for multi-modality remote sensing classification

CFNet: A Cross Fusion Network for Joint Land Cover Classification Using Optical and SAR Images

MHFNet: an Improved HGR Multimodal Network for Informative Correlation Fusion in Remote Sensing Image Classification

Hyperspectral and SAR Image Classification via Multiscale Interactive Fusion Network

An Adaptive Multilayer Feature Fusion Strategy for Remote Sensing Scene Classification

Dynamic Convolution Covariance Network Using Multiscale Feature Fusion for Remote Sensing Scene Image Classification

A Semantically Nonredundant Continuous-Scale Feature Network for Panchromatic and Multispectral Classification

A multimodal hyper-fusion transformer for remote sensing image classification

Statistical Texture Awareness Network for Hyperspectral Image Classification

A Spatial-Spectral Relation-Guided Fusion Network for Multisource Optical RS Image Classification

HDTFF-Net: Hierarchical Deep Texture Features Fusion Network for High-resolution Remote Sensing Scene Classification

Remote Sensing Scene Classification Using Heterogeneous Feature Extraction and Multi-Level Fusion