Abstract:Recently, the scenes in large high-resolution remote sensing (HRRS) datasets have been classified using convolutional neural network (CNN)-based methods. Such methods are well-suited for spatial feature extraction and can classify images with relatively high accuracy. However, CNNs do not adequately learn the long-distance dependencies between images and features in image processing, despite this being necessary for HRRS image processing as the semantic content of the scenes in these images is closely related to their spatial relationship. CNNs also have limitations in solving problems related to large intra-class differences and high inter-class similarity. To overcome these challenges, in this study we combine the channel-spatial attention (CSA) mechanism with the Vision Transformer method to propose an effective HRRS image scene classification framework using Channel-Spatial Attention Transformers (CSAT). The proposed model extracts the channel and spatial features of HRRS images using CSA and the Multi-head Self-Attention (MSA) mechanism in the transformer module. First, the HRRS image is mapped into a series of multiple planar 2D patch vectors after passing to the CSA. Second, the ordered vector is obtained via the linear transformation of each vector, and the position and learnable embedding vectors are added to the sequence vector to capture the inter-feature dependencies at a distance from the generated image. Next, we use MSA to extract image features and the residual network structure to complete the encoder construction to solve the gradient disappearance problem and avoid overfitting. Finally, a multi-layer perceptron is used to classify the scenes in the HRRS images. The CSAT network is evaluated using three public remote sensing scene image datasets: UC-Merced, AID, and NWPU-RESISC45. The experimental results show that the proposed CSAT network outperforms a selection of state-of-the-art methods in terms of scene classification.

FCT: Fusing CNN and Transformer for Scene Classification

A Novel Transformer Network with a CNN-Enhanced Cross-Attention Mechanism for Hyperspectral Image Classification

When CNNs Meet Vision Transformer: A Joint Framework for Remote Sensing Scene Classification

SemiCVT: Semi-Supervised Convolutional Vision Transformer for Semantic Segmentation

CTA-Net: A CNN-Transformer Aggregation Network for Improving Multi-Scale Feature Extraction

CTMFNet: CNN and Transformer Multiscale Fusion Network of Remote Sensing Urban Scene Imagery

CCTSS: the Combination of CNN and Transformer with Shared Sublayer for Detection and Classification

CNN and Transformer Fusion for Remote Sensing Image Semantic Segmentation

Scene Classification Based on Two-Stage Deep Feature Fusion

TCNet: Multiscale Fusion of Transformer and CNN for Semantic Segmentation of Remote Sensing Images

Fusion of Dilated Convolution in CNN and Transformer Networks for Hyperspectral Image Classification

SwinHCST: a deep learning network architecture for scene classification of remote sensing images based on improved CNN and Transformer

Resformer: Bridging Residual Network and Transformer for Remote Sensing Scene Classification

Dense Cross-Scale Transformer with Channel Learning for Remote Sensing Scene Classification

Transformer based on channel-spatial attention for accurate classification of scenes in remote sensing image

A Novel Hybrid Transformer-Cnn Architecture for Environmental Microorganism Classification.

CTIF-Net: A CNN-Transformer Iterative Fusion Network for Salient Object Detection

IFTSDNet: An Interact-Feature Transformer Network With Spatial Detail Enhancement Module for Change Detection

Convolution-Transformer Adaptive Fusion Network for Hyperspectral Image Classification

DctViT: Discrete Cosine Transform Meet Vision Transformers

CMT: Convolutional Neural Networks Meet Vision Transformers