Abstract:Convolutional neural networks (CNNs) have dominated the hyperspectral image (HSI) classification due to their tremendous feature learning capability. However, the formidable local sensitivity is both a strength and a weakness. Recently, the vision transformers have exhibited impressive performances on various vision problems. Compared with CNNs, they can model long-range dependencies to learn more abundant interactions between spatial locations. Nevertheless, the existing transformer-based HSI classification methods also concentrate too much on the advantages of the transformer architecture and disregard the importance of local dependencies. In addition, token generation and token mixers in transformer-like architectures have not been adequately explored, leading to difficulties in obtaining the best classification performance. To deal with these problems, a novel multiscale hierarchical conv-aided Fourierformer (MHCFormer) is proposed for HSI classification. To the best of our knowledge, this is the first time that CNN, transformer, and Fourier transform are skillfully combined for HSI classification. The proposed MHCFormer involves three stages, i.e., multiscale spectral–spatial token generation, hierarchical token learning, and a classification head. The multiscale spectral–spatial token generation is constructed to transform HSI into tokens with multiscale-enhanced spectral–spatial information. The hierarchical token learning is designed to explore multiscale tokens globally and locally by integrating the design philosophy of transformers and CNNs along with Fourier transforms into a block and stacking the blocks hierarchically. Extensive experimental results on the new WHU-Hi-HanChuan dataset and the widely used Indian Pines and Houston 2013 datasets have demonstrated the superiority of MHCFormer over other state-of-the-art methods. The code of our work will be available publicly at https://github.com/Tikiten/MHCFormer.

WaveFormer: Spectral–Spatial Wavelet Transformer for Hyperspectral Image Classification

WaveMamba: Spatial-Spectral Wavelet Mamba for Hyperspectral Image Classification

SWFormer: Stochastic Windows Convolutional Transformer for Hybrid Modality Hyperspectral Classification

Pyramid Hierarchical Transformer for Hyperspectral Image Classification

A hybrid convolution transformer for hyperspectral image classification

SpectralFormer: Rethinking Hyperspectral Image Classification With Transformers

Adaptive Learnable Spectral–Spatial Fusion Transformer for Hyperspectral Image Classification

Hyperspectral Image Classification Using Spectral–Spatial Token Enhanced Transformer with Hash-Based Positional Embedding

Unformer: A Transformer-Based Approach for Adaptive Multi-Scale Feature Aggregation in Underwater Image Enhancement

MHCFormer: Multiscale Hierarchical Conv-Aided Fourierformer for Hyperspectral Image Classification

Transformers Fusion across Disjoint Samples for Hyperspectral Image Classification

Hyperspectral Image Transformer Classification Networks

MASSFormer: Memory-Augmented Spectral-Spatial Transformer for Hyperspectral Image Classification

MultiScale Spectral-Spatial Convolutional Transformer for Hyperspectral Image Classification

MultiScale spectral–spatial convolutional transformer for hyperspectral image classification

Spectral-Swin Transformer with Spatial Feature Extraction Enhancement for Hyperspectral Image Classification

Hierarchical Attention Transformer for Hyperspectral Image Classification

An Enhanced 3-D Discrete Wavelet Transform for Hyperspectral Image Classification

Expeditious Hyperspectral Image Classification With Inner and Outer Layered Transformer Using Feature Enhancement

Cascaded Convolution-Based Transformer With Densely Connected Mechanism for Spectral–Spatial Hyperspectral Image Classification