Abstract:In recent years, convolutional neural network (CNN)-based methods have been widely used for remote sensing (RS) scene classification tasks and have achieved excellent results. However, CNNs are not good at exploring contextual information, which is essential for fully understanding RS scenes. A new model named transformer attracts researchers' attention to address this problem, which is skilled in mining the latent contextual information in RS scenes. Nevertheless, since the contents of RS scenes are diverse in type and various in scale, the performance of the original transformer in RS scene classification cannot reach what we expect. In addition, due to the specific self-attention mechanism, the time costs of the transformer are high, which hinders its practicability in the RS community. To overcome the above limitations, we propose a new model named efficient multiscale transformer and cross-level attention learning (EMTCAL) for RS scene classification in this article. EMTCAL combines the advantages of CNN and transformer to mine information within RS scenes fully. First, it uses a multilayer feature extraction module (MFEM) to acquire global visual features and multilevel convolutional features from RS scenes. Second, a contextual information extraction module (CIEM) is proposed to capture rich contextual information from multilevel features. In CIEM, taking the characteristics of RS scenes and the computational complexity into account, we propose an efficient multiscale transformer (EMST). EMST can mine the abundant knowledge with various scales hidden in RS scenes and model their inherent relations at small time costs. Third, a cross-level attention module (CLAM) is developed to aggregate and explore correlations of multilevel features. Finally, a class score fusion module (CSFM) is designed to integrate the contributions of global and aggregated multilevel features for discriminative scene representations. Extensive experiments are conduct- d on three public RS scene datasets. The positive results demonstrate that our EMTCAL can achieve superior classification performance and outperform many state-of-the-art methods. Our source codes are available in https://github.com/TangXu-Group/Remote-Sensing-Images-Classification/tree/main/EMTCAL.

Transformer-based Contrastive Prototypical Clustering for Multimodal Remote Sensing Data.

MCFT: Multimodal Contrastive Fusion Transformer for Classification of Hyperspectral Image and LiDAR Data

Remote Sensing Collaborative Classification Using Multimodal Adaptive Modulation Network

Deep Symmetric Fusion Transformer for Multimodal Remote Sensing Data Classification

Multimodal Fusion Transformer for Remote Sensing Image Classification

A multimodal hyper-fusion transformer for remote sensing image classification

Fast Projected Fuzzy Clustering with Anchor Guidance for Multimodal Remote Sensing Imagery

Learning transformer-based heterogeneously salient graph representation for multimodal remote sensing image classification

Unsupervised remote sensing image classification with differentiable feature clustering by coupled transformer

Transformer-Based Incomplete Multi-Modal Learning for Land Cover Classification

TCPSNet: Transformer and Cross-Pseudo-Siamese Learning Network for Classification of Multi-Source Remote Sensing Images

Transformer-based Multi-Modal Learning for Multi Label Remote Sensing Image Classification

Cross-Modal Contrastive Learning for Remote Sensing Image Classification.

EMTCAL: Efficient Multiscale Transformer and Cross-Level Attention Learning for Remote Sensing Scene Classification

Multimodal Transformer Using Cross-Channel attention for Object Detection in Remote Sensing Images

Convolutional Neural Networks for Multimodal Remote Sensing Data Classification

Multicrop Fusion Strategy Based on Prototype Assignment for Remote Sensing Image Scene Classification.

Tri-CLT: Learning Tri-Modal Representations with Contrastive Learning and Transformer for Multimodal Sentiment Recognition

DKDFN: Domain Knowledge-Guided deep collaborative fusion network for multimodal unitemporal remote sensing land cover classification

Triplet Contrastive Learning Framework With Adversarial Hard-Negative Sample Generation for Multimodal Remote Sensing Images

Learning rich multimodal representation for robust land cover classification in fog