Abstract:In the past, convolutional neural network (CNN) has become one of the most popular deep learning frameworks, and has been widely used in Hyperspectral image classification tasks. Convolution (Conv) in CNN uses filter weights to extract features in local receiving domain, and the weight parameters are shared globally, which more focus on the high‐frequency information of the image. Different from Conv, Transformer can obtain the long‐term dependence between long‐distance features through modelling, and adaptively focus on different regions. In addition, Transformer is considered as a low‐pass filter, which more focuses on the low‐frequency information of the image. Considering the complementary characteristics of Conv and Transformer, the two modes can be integrated for full feature extraction. In addition, the most important image features correspond to the discrimination region, while the secondary image features represent important but easily ignored regions, which are also conducive to the classification of HSIs. In this study, a complementary integrated Transformer network (CITNet) for hyperspectral image classification is proposed. Firstly, three‐dimensional convolution (Conv3D) and two‐dimensional convolution (Conv2D) are utilised to extract the shallow semantic information of the image. In order to enhance the secondary features, a channel Gaussian modulation attention module is proposed, which is embedded between Conv3D and Conv2D. This module can not only enhance secondary features, but suppress the most important and least important features. Then, considering the different and complementary characteristics of Conv and Transformer, a complementary integrated Transformer module is designed. Finally, through a large number of experiments, this study evaluates the classification performance of CITNet and several state‐of‐the‐art networks on five common datasets. The experimental results show that compared with these classification networks, CITNet can provide better classification performance.

CCTSS: the Combination of CNN and Transformer with Shared Sublayer for Detection and Classification

A Novel Transformer Network with a CNN-Enhanced Cross-Attention Mechanism for Hyperspectral Image Classification

CTA-Net: A CNN-Transformer Aggregation Network for Improving Multi-Scale Feature Extraction

CTIF-Net: A CNN-Transformer Iterative Fusion Network for Salient Object Detection

Combining transformer global and local feature extraction for object detection

FCT: Fusing CNN and Transformer for Scene Classification

Relating CNN-Transformer Fusion Network for Change Detection

CCTNet: CNN and Cross-Shaped Transformer Hybrid Network for Remote Sensing Image Semantic Segmentation

A CNN-Transformer Network Combining CBAM for Change Detection in High-Resolution Remote Sensing Images

ConvTransNet: A CNN–Transformer Network for Change Detection With Multiscale Global–Local Representations

CNN-transformer mixed model for object detection

CTC-Net: A Novel Coupled Feature-Enhanced Transformer and Inverted Convolution Network for Medical Image Segmentation

MCTNet: A Multi-Scale CNN-Transformer Network for Change Detection in Optical Remote Sensing Images

3DCTN: 3D Convolution-Transformer Network for Point Cloud Classification

CTCNet: A CNN Transformer Capsule Network for Sleep Stage Classification

Deep global-local transformer network combined with extended morphological profiles for hyperspectral image classification

When CNNs Meet Vision Transformer: A Joint Framework for Remote Sensing Scene Classification

CCTNet: Coupled CNN and Transformer Network for Crop Segmentation of Remote Sensing Images.

CNN and Transformer interaction network for hyperspectral image classification

Transform-Invariant Convolutional Neural Networks for Image Classification and Search

A Complementary Integrated Transformer Network for Hyperspectral Image Classification