Abstract:Convolutional neural networks (CNNs) have dominated the hyperspectral image (HSI) classification due to their tremendous feature learning capability. However, the formidable local sensitivity is both a strength and a weakness. Recently, the vision transformers have exhibited impressive performances on various vision problems. Compared with CNNs, they can model long-range dependencies to learn more abundant interactions between spatial locations. Nevertheless, the existing transformer-based HSI classification methods also concentrate too much on the advantages of the transformer architecture and disregard the importance of local dependencies. In addition, token generation and token mixers in transformer-like architectures have not been adequately explored, leading to difficulties in obtaining the best classification performance. To deal with these problems, a novel multiscale hierarchical conv-aided Fourierformer (MHCFormer) is proposed for HSI classification. To the best of our knowledge, this is the first time that CNN, transformer, and Fourier transform are skillfully combined for HSI classification. The proposed MHCFormer involves three stages, i.e., multiscale spectral–spatial token generation, hierarchical token learning, and a classification head. The multiscale spectral–spatial token generation is constructed to transform HSI into tokens with multiscale-enhanced spectral–spatial information. The hierarchical token learning is designed to explore multiscale tokens globally and locally by integrating the design philosophy of transformers and CNNs along with Fourier transforms into a block and stacking the blocks hierarchically. Extensive experimental results on the new WHU-Hi-HanChuan dataset and the widely used Indian Pines and Houston 2013 datasets have demonstrated the superiority of MHCFormer over other state-of-the-art methods. The code of our work will be available publicly at https://github.com/Tikiten/MHCFormer.

Hyperspectral Image Classification based on Multi-Scale Convolutional Features and Multi-Attention Mechanisms

A Novel Transformer Network with a CNN-Enhanced Cross-Attention Mechanism for Hyperspectral Image Classification

Multiscale 3-D-2-D Mixed CNN and Lightweight Attention-Free Transformer for Hyperspectral and LiDAR Classification

Multi-Scale Residual Spectral–Spatial Attention Combined with Improved Transformer for Hyperspectral Image Classification

An Effective Hyperspectral Image Classification Network Based on Multi-Head Self-Attention and Spectral-Coordinate Attention

MHIAIFormer: Multihead Interacted and Adaptive Integrated Transformer With Spatial-Spectral Attention for Hyperspectral Image Classification

MHIAIFormer: Multi-Head Interacted and Adaptive Integrated Transformer with Spatial-Spectral Attention for Hyperspectral Image Classification

Subpixel Multilevel Scale Feature Learning and Adaptive Attention Constraint Fusion for Hyperspectral Image Classification

Hierarchical Multi-Scale Convolutional Neural Networks For Hyperspectral Image Classification

A Dual-Branch Multiscale Transformer Network for Hyperspectral Image Classification

Hyperspectral Image Classification Based on Multibranch Attention Transformer Networks

Multiscale Pixel-Level and Superpixel-Level Method for Hyperspectral Image Classification: Adaptive Attention and Parallel Multi-Hop Graph Convolution

Center-Highlighted Multiscale CNN for Classification of Hyperspectral Images

Multiscale Densely Connected Attention Network for Hyperspectral Image Classification

HAMNet: hyperspectral image classification based on hybrid neural network with attention mechanism and multi-scale feature fusion

Hyperspectral image classification based on multiscale piecewise spectral-spatial attention network

Cnn-assisted multi-hop graph attention network for hyperspectral image classification

Attention Multibranch Convolutional Neural Network for Hyperspectral Image Classification Based on Adaptive Region Search

MHCFormer: Multiscale Hierarchical Conv-Aided Fourierformer for Hyperspectral Image Classification

Multiscale Feature Fusion Network Incorporating 3D Self-Attention for Hyperspectral Image Classification

Multi-Scale Spatial–Spectral Residual Attention Network for Hyperspectral Image Classification