Abstract:Remote sensing (RS) scene classification plays an essential role in the RS community and has attracted increasing attention due to its wide applications. Recently, benefiting from the powerful feature learning capabilities of convolutional neural networks (CNNs), the accuracy of the RS scene classification has significantly been improved. Although the existing CNN-based methods achieve excellent results, there is still room for improvement. First, the CNN-based methods are adept at capturing the global information from RS scenes. Still, the context relationships hidden in RS scenes cannot be thoroughly mined. Second, due to the specific structure, it is easy for normal CNNs to exploit the heterogenous information from RS scenes. Nevertheless, the homogenous information, which is also crucial to comprehensively understand complex contents within RS scenes, does not get the attention it deserves. Third, most CNNs focus on establishing the relationships between RS scenes and semantic labels. However, the similarities between them are not considered deeply, which are helpful to distinguish the intra-/interclass samples. To overcome the limitations mentioned previously, we propose a homoheterogenous transformer learning (HHTL) framework for the RS scene classification in this article. First, a patch generation module is designed to generate homogenous and heterogenous patches. Then, a dual-branch feature learning module (FLM) is proposed to mine homogenous and heterogenous information within RS scenes simultaneously. In the FLM, based on vision transformer, not only the global information but also the local areas and their context information can be captured. Finally, we design a classification module, which consists of a fusion submodule and a metric-learning module. It can integrate homoheterogenous information and compact/separate samples from the same/different RS scene categories. Extensive experiments are conducted on four public RS scene datasets. The encouraging results demonstrate that our HHTL framework can outperform many state-of-the-art methods. Our source codes are available at the below website.

Convolutional Neural Network Based Heterogeneous Transfer Learning for Remote-Sensing Scene Classification

Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery

Scene Classification of High-Resolution Remote Sensing Image Using Transfer Learning with Multi-model Feature Extraction Framework.

Heterogeneous Transfer Learning for Hyperspectral Image Classification Based on Convolutional Neural Network

Fusing Deep Local And Global Features For Remote Sensing Image Scene Classification

Heterogeneous feature learning network for multimodal remote sensing image collaborative classification

Transferring CNN With Adaptive Learning for Remote Sensing Scene Classification

Remote Sensing Scene Classification Using Sparse Representation-Based Framework with Deep Feature Fusion

Fusing Deep Features by Kernel Collaborative Representation for Remote Sensing Scene Classification

Homo–Heterogenous Transformer Learning Framework for RS Scene Classification

Combing Triple-Part Features of Convolutional Neural Networks for Scene Classification in Remote Sensing

Extracting feature fusion and co-saliency clusters using transfer learning techniques for improving remote sensing scene classification

Adaptively heterogeneous transfer learning for hyperspectral image classification

Cross-domain Residual Deep NMF for Transfer Learning Between Different Hyperspectral Image Scenes.

Aggregating Rich Hierarchical Features for Scene Classification in Remote Sensing Imagery

Feature and Model Level Fusion of Pretrained CNN for Remote Sensing Scene Classification

An Efficient and Lightweight Convolutional Neural Network for Remote Sensing Image Scene Classification

Domain‐invariant Attention Network for Transfer Learning Between Cross‐scene Hyperspectral Images

When Self-Supervised Learning Meets Scene Classification: Remote Sensing Scene Classification Based on a Multitask Learning Framework

Heterogeneous Spectral-Spatial Feature Transfer With Structure Preserved Distribution Alignment for Hyperspectral Image Classification

Scene Classification Of High Resolution Remote Sensing Images Using Convolutional Neural Networks