Abstract:In recent years, convolutional neural network (CNN)-based methods have been widely used for remote sensing (RS) scene classification tasks and have achieved excellent results. However, CNNs are not good at exploring contextual information, which is essential for fully understanding RS scenes. A new model named transformer attracts researchers' attention to address this problem, which is skilled in mining the latent contextual information in RS scenes. Nevertheless, since the contents of RS scenes are diverse in type and various in scale, the performance of the original transformer in RS scene classification cannot reach what we expect. In addition, due to the specific self-attention mechanism, the time costs of the transformer are high, which hinders its practicability in the RS community. To overcome the above limitations, we propose a new model named efficient multiscale transformer and cross-level attention learning (EMTCAL) for RS scene classification in this article. EMTCAL combines the advantages of CNN and transformer to mine information within RS scenes fully. First, it uses a multilayer feature extraction module (MFEM) to acquire global visual features and multilevel convolutional features from RS scenes. Second, a contextual information extraction module (CIEM) is proposed to capture rich contextual information from multilevel features. In CIEM, taking the characteristics of RS scenes and the computational complexity into account, we propose an efficient multiscale transformer (EMST). EMST can mine the abundant knowledge with various scales hidden in RS scenes and model their inherent relations at small time costs. Third, a cross-level attention module (CLAM) is developed to aggregate and explore correlations of multilevel features. Finally, a class score fusion module (CSFM) is designed to integrate the contributions of global and aggregated multilevel features for discriminative scene representations. Extensive experiments are conduct- d on three public RS scene datasets. The positive results demonstrate that our EMTCAL can achieve superior classification performance and outperform many state-of-the-art methods. Our source codes are available in https://github.com/TangXu-Group/Remote-Sensing-Images-Classification/tree/main/EMTCAL.

Remote Sensing Image Classification Based on Non-Linear Enhanced Attention Mechanism

A Novel Transformer Network with a CNN-Enhanced Cross-Attention Mechanism for Hyperspectral Image Classification

Remote Sensing Scene Image Classification Model Based on Multi-Scale Features and Attention Mechanism

A multimodal hyper-fusion transformer for remote sensing image classification

Improved Attention Mechanism and Residual Network for Remote Sensing Image Scene Classification

FAM: Improving columnar vision transformer with feature attention mechanism

A Multi-Branch Feature Fusion Strategy Based on an Attention Mechanism for Remote Sensing Image Scene Classification

A Bio-Inspired Visual Perception Transformer for Cross-Domain Semantic Segmentation of High-Resolution Remote Sensing Images

Vision Transformers for Remote Sensing Image Classification

Attention-Mechanism-Containing Neural Networks for High-Resolution Remote Sensing Image Classification

Hybrid Attention Fusion Embedded in Transformer for Remote Sensing Image Semantic Segmentation

Hyperspectral Image Classification Based on Multibranch Attention Transformer Networks

Transformer based on channel-spatial attention for accurate classification of scenes in remote sensing image

Agent Attention: On the Integration of Softmax and Linear Attention

Multi-Scale Residual Spectral–Spatial Attention Combined with Improved Transformer for Hyperspectral Image Classification

EMTCAL: Efficient Multiscale Transformer and Cross-Level Attention Learning for Remote Sensing Scene Classification

Automated classification of remote sensing satellite images using deep learning based vision transformer

Remote Sensing Image Classification Based on a Cross-Attention Mechanism and Graph Convolution

Spectral–Spatial Feature Extraction for Hyperspectral Image Classification Using Enhanced Transformer with Large-Kernel Attention

Hyperspectral Remote-Sensing Classification Combining Transformer and Multiscale Residual Mechanisms