Abstract:Recently, vision transformer (ViT)-based deep learning (DL) models have achieved remarkable performance gains in hyperspectral image classification (HSIC) due to their abilities to model long-range dependencies and extract global spatial features. However, ViT is built with a stack of Transformer blocks and faces the challenge of learning a large number of parameters when processing hyperspectral data. Besides, the inherent modeling of global correlation in Transformer ignores the effective representation of local spatial and spectral features. To address these issues, we propose a lightweight ViT network known as groupwise separable convolutional ViT (GSC-ViT). First, a groupwise separable convolution (GSC) module, which is a combination of grouped pointwise convolution (GPWC) and group convolution, is designed to significantly decrease the number of convolutional kernel parameters, and effectively capture local spectral–spatial information in hyperspectral image (HSI). Second, a groupwise separable multihead self-attention (GSSA) module is employed to substitute the conventional multihead self-attention (MSA) in ViT, in which the groupwise self-attention (GSA) provides local spatial feature extraction, and the pointwise self-attention (PWSA) provides global spatial feature extraction. Third, a simple pointwise layer with enhanced skip connection mechanism is employed to substitute the multilayer perceptron (MLP) layer in all Transformer blocks of ViT, so as to eliminate unnecessary nonlinear transformations and facilitate the fusion of features derived from GSC and GSSA modules. Extensive experiments on four benchmark hyperspectral datasets reveal that our GSC-ViT can achieve surprising classification performance with relatively few training samples as compared with some existing HSIC approaches. The source code is available at https://github.com/flyzzie/TGRS-GSC-VIT.

Vision Transformer With Contrastive Learning for Hyperspectral Image Classification

Hyperspectral Image Classification Using Groupwise Separable Convolutional Vision Transformer Network

Hybrid Conv-ViT Network for Hyperspectral Image Classification

Discriminative Vision Transformer for Heterogeneous Cross-Domain Hyperspectral Image Classification

Hybrid Vision Transformer Model for Hyperspectral Image Classification.

Vision Transformer With Contrastive Learning for Remote Sensing Image Scene Classification

Cross-Domain Hyperspectral Image Classification Based on Transformer

3D-Convolution Guided Spectral-Spatial Transformer for Hyperspectral Image Classification

Tripartite‐structure transformer for hyperspectral image classification

Multi-granularity Vision Transformer Via Semantic Token for Hyperspectral Image Classification

Vision Transformer for Contrastive Clustering

A Vision Transformer for Fine-Grained Classification by Reducing Noise and Enhancing Discriminative Information

CSiT: A Multiscale Vision Transformer for Hyperspectral Image Classification.

Contrastive Learning Based on Transformer for Hyperspectral Image Classification

Vision Transformer-Based Ensemble Learning for Hyperspectral Image Classification

Joint Multi-Scale CNN and Vision Transformer for Hyperspectral Image Classification

A Spatial–Spectral Transformer for Hyperspectral Image Classification Based on Global Dependencies of Multi-Scale Features

Light Self-Gaussian-Attention Vision Transformer for Hyperspectral Image Classification.

SCViT: A Spatial-Channel Feature Preserving Vision Transformer for Remote Sensing Image Scene Classification

ConVaT: A Variational Generative Transformer With Momentum Contrastive Learning for Hyperspectral Image Classification

An Efficient Transformer with Neighborhood Contrastive Tokenization for Hyperspectral Images Classification