Abstract:Recently, due to the powerful capability at modeling the long-range relationships, Transformer-based methods have been widely explored in many research areas, including hyperspectral image (HSI) classification. However, because of lots of trainable parameters and the lack of inductive bias, it is difficult to train a Transformer-based HSI classifier, especially when the number of training samples is limited. To address this issue, in this study, spectral-spatial masked Transformer (SS-MTr) is explored for HSI classification, which uses a two-stage training strategy. In the first stage, SS-MTr pretrains a vanilla Transformer via reconstruction from masked HSI inputs, which embeds the local inductive bias into the Transformer. In the second stage, the well pretrained Transformer is cooperated with a fully connected layer and is then fine-tuned for the HSI classification. Furthermore, in order to incorporate discriminative feature learning into the SS-MTr, three SS-MTr-based methods, including contrastive SS-MTr (C-SS-MTr), supervised SS-MTr (S-SS-MTr), and supervised C-SS-MTr (SC-SS-MTr), are proposed by adding extra branches for specific tasks in parallel with the existing reconstruction task. Specifically, the proposed C-SS-MTr adds a contrastive loss, which brings instance discriminability. Besides, the proposed S-SS-MTr builds an extra classification branch for embracing interclass discriminability and intraclass similarity. Moreover, the proposed SC-SS-MTr combines C-SS-MTr and S-SS-MTr for better generalization. The proposed SS-MTr, C-SS-MTr, S-SS-MTr, and SC-SS-MTr are tested on three popular hyperspectral datasets (i.e., Indian Pines, Pavia University, and Houston). The obtained results reveal that the proposed models achieve competitive results compared with the state-of-the-art HSI classification methods. The code is available at https://github.com/mengduanjinghua/SS-MTr.

Semi-Supervised Co-Training Model Using Convolution and Transformer for Hyperspectral Image Classification

Semi-supervised Co-training Model Using Convolution and Transformer for Hyperspectral Image Classifica Tion

Hyperspectral Image Classification Using Hierarchical Spatial-Spectral Transformer

Hybrid Spectral-Spatial Convolutional Network and Transformer with Mixup Regularization for Hyperspectral Image Classification

Multiscale and Cross-Level Attention Learning for Hyperspectral Image Classification

MSMT-LCL: Multiscale Spatial-Spectral Masked Transformer With Local Contrastive Learning for Hyperspectral Image Classification

Multi-scale Spectral-Spatial Dual-Transformer Network for Hyperspectral Image Classification

CTF-SSCL: CNN-Transformer for Few-Shot Hyperspectral Image Classification Assisted by Semisupervised Contrastive Learning

Hyperspectral Image Classification based on Multi-Scale Convolutional Features and Multi-Attention Mechanisms

Semi-Active Convolutional Neural Networks for Hyperspectral Image Classification

CNN and Transformer Hybrid Network for Hyperspectral Image Classification

Hyperspectral Remote-Sensing Classification Combining Transformer and Multiscale Residual Mechanisms

Main–Sub Transformer with Spectral–Spatial Separable Convolution for Hyperspectral Image Classification

Grouped Bidirectional LSTM Network and Multi-Stage Fusion Convolutional Transformer for Hyperspectral Image Classification

MSCC-ViT:A Multiscale Visual-Transformer Network Using Convolution Crossing Attention for Hyperspectral Unmixing

Hyperspectral Image Classification Based on Multibranch Attention Transformer Networks

Spectral–Spatial Masked Transformer with Supervised and Contrastive Learning for Hyperspectral Image Classification

Deep Spectral Spatial Feature Enhancement Through Transformer for Hyperspectral Image Classification

ELS2T: Efficient Lightweight Spectral-Spatial Transformer for Hyperspectral Image Classification

A Global+ Multiscale Hybrid Network for Hyperspectral Image Classification

End-to-End Convolutional Network and Spectral-Spatial Transformer Architecture for Hyperspectral Image Classification