Abstract:Remote sensing (RS) scene classification plays an essential role in the RS community and has attracted increasing attention due to its wide applications. Recently, benefiting from the powerful feature learning capabilities of convolutional neural networks (CNNs), the accuracy of the RS scene classification has significantly been improved. Although the existing CNN-based methods achieve excellent results, there is still room for improvement. First, the CNN-based methods are adept at capturing the global information from RS scenes. Still, the context relationships hidden in RS scenes cannot be thoroughly mined. Second, due to the specific structure, it is easy for normal CNNs to exploit the heterogenous information from RS scenes. Nevertheless, the homogenous information, which is also crucial to comprehensively understand complex contents within RS scenes, does not get the attention it deserves. Third, most CNNs focus on establishing the relationships between RS scenes and semantic labels. However, the similarities between them are not considered deeply, which are helpful to distinguish the intra-/interclass samples. To overcome the limitations mentioned previously, we propose a homoheterogenous transformer learning (HHTL) framework for the RS scene classification in this article. First, a patch generation module is designed to generate homogenous and heterogenous patches. Then, a dual-branch feature learning module (FLM) is proposed to mine homogenous and heterogenous information within RS scenes simultaneously. In the FLM, based on vision transformer, not only the global information but also the local areas and their context information can be captured. Finally, we design a classification module, which consists of a fusion submodule and a metric-learning module. It can integrate homoheterogenous information and compact/separate samples from the same/different RS scene categories. Extensive experiments are conducted on four public RS scene datasets. The encouraging results demonstrate that our HHTL framework can outperform many state-of-the-art methods. Our source codes are available at the below website.

Iterative Reweighting Heterogeneous Transfer Learning Framework for Supervised Remote Sensing Image Classification

Recurrent Homography Estimation Using Homography-Guided Image Warping and Focus Transformer

Unsupervised Transfer Learning Using For Multi-Model Remote Sensing Data Classification

Joint Feature-Space and Sample-Space Based Heterogeneous Feature Transfer Method for Object Recognition Using Remote Sensing Images with Different Spatial Resolutions

A General Transitive Transfer Learning Framework for Cross-Optical Sensor Remote Sensing Image Scene Understanding

Cross-domain Residual Deep NMF for Transfer Learning Between Different Hyperspectral Image Scenes.

Joint spectral-spatial hyperspectral classification based on transfer learning (SSTL) from red-green-blue (RGB) images

Dictionary Learning-Based Feature-Level Domain Adaptation for Cross-Scene Hyperspectral Image Classification

Robust Self-Paced Incremental Learning for Multitemporal Remote Sensing Image Classification

Homo–Heterogenous Transformer Learning Framework for RS Scene Classification

Active-Learning-Incorporated Deep Transfer Learning for Hyperspectral Image Classification

An Unsupervised Domain Adaptation Method for Multi-Modal Remote Sensing Image Classification

A Trusted Generative-Discriminative Joint Feature Learning Framework for Remote Sensing Image Classification

Re-Equipping Residual Networks for Few-Shot Hyperspectral Image Classification

Exploring Hard Samples in Multiview for Few-Shot Remote Sensing Scene Classification

Transferring CNN With Adaptive Learning for Remote Sensing Scene Classification

Transfer Learning-Based Hyperspectral Image Classification Using Residual Dense Connection Networks

From Video to Hyperspectral: Hyperspectral Image-Level Feature Extraction with Transfer Learning

Incremental Learning With Open-Set Recognition for Remote Sensing Image Scene Classification

High-Resolution Remote Sensing Imagery Classification Of Imbalanced Data Using Multistage Sampling Method And Deep Neural Networks