Abstract:In recent years, convolutional neural network (CNN)-based methods have been widely used for remote sensing (RS) scene classification tasks and have achieved excellent results. However, CNNs are not good at exploring contextual information, which is essential for fully understanding RS scenes. A new model named transformer attracts researchers' attention to address this problem, which is skilled in mining the latent contextual information in RS scenes. Nevertheless, since the contents of RS scenes are diverse in type and various in scale, the performance of the original transformer in RS scene classification cannot reach what we expect. In addition, due to the specific self-attention mechanism, the time costs of the transformer are high, which hinders its practicability in the RS community. To overcome the above limitations, we propose a new model named efficient multiscale transformer and cross-level attention learning (EMTCAL) for RS scene classification in this article. EMTCAL combines the advantages of CNN and transformer to mine information within RS scenes fully. First, it uses a multilayer feature extraction module (MFEM) to acquire global visual features and multilevel convolutional features from RS scenes. Second, a contextual information extraction module (CIEM) is proposed to capture rich contextual information from multilevel features. In CIEM, taking the characteristics of RS scenes and the computational complexity into account, we propose an efficient multiscale transformer (EMST). EMST can mine the abundant knowledge with various scales hidden in RS scenes and model their inherent relations at small time costs. Third, a cross-level attention module (CLAM) is developed to aggregate and explore correlations of multilevel features. Finally, a class score fusion module (CSFM) is designed to integrate the contributions of global and aggregated multilevel features for discriminative scene representations. Extensive experiments are conduct- d on three public RS scene datasets. The positive results demonstrate that our EMTCAL can achieve superior classification performance and outperform many state-of-the-art methods. Our source codes are available in https://github.com/TangXu-Group/Remote-Sensing-Images-Classification/tree/main/EMTCAL.

Tensor Canonical Correlation Analysis Networks for Multi-view Remote Sensing Scene Recognition

Parallel-Connected Residual Channel Attention Network for Remote Sensing Image Super-Resolution

Dynamic Convolution Covariance Network Using Multi-Scale Feature Fusion for Remote Sensing Scene Image Classification

Tensor Canonical Correlation Analysis for Multi-View Dimension Reduction

TCANet: Triple Context-Aware Network for Weakly Supervised Object Detection in Remote Sensing Images

Scene Change Detection VIA Deep Convolution Canonical Correlation Analysis Neural Network

Effective Multiscale Residual Network With High-Order Feature Representation for Optical Remote Sensing Scene Classification

Multi-Scale and Multi-Network Deep Feature Fusion for Discriminative Scene Classification of High-Resolution Remote Sensing Images

Dynamic Convolution Covariance Network Using Multiscale Feature Fusion for Remote Sensing Scene Image Classification

A multi-scale dense residual correlation network for remote sensing scene classification

Sparse canonical correlation analysis for recognition

EMTCAL: Efficient Multiscale Transformer and Cross-Level Attention Learning for Remote Sensing Scene Classification

Remote Sensing Image Change Detection Network Based on Twin High-Resolution Representation

Remote Sensing Object Detection with Feature-Associated Convolutional Neural Networks

On the Selective and Invariant Representation of DCNN for High-Resolution Remote Sensing Image Recognition

Relation-Attention Networks for Remote Sensing Scene Classification

Spatial Information Considered Network for Scene Classification

Remote Sensing Image Super-Resolution Using Enriched Spatial-Channel Feature Aggregation Networks

Siamese Convolutional Neural Networks for Remote Sensing Scene Classification

Kolmogorov-Arnold Network for Satellite Image Classification in Remote Sensing

Asymmetric Network Combining CNN and Transformer for Building Extraction from Remote Sensing Images