Abstract:Remote sensing image scene classification is to annotate semantic categories for image areas covering multiple land cover types, reflecting the spatial aggregation of relevant social resources among feature objects, which is one of the remote sensing interpretation tasks with higher challenges for algorithms to understand the images. Nowadays, scene semantic information extraction of images using deep neural networks is also one of the hot research directions. In comparison to other algorithms, deep neural networks can better capture semantic information in images to achieve higher classification accuracy involved in applications such as urban planning. In recent years, multi-modal models represented by image-text have achieved satisfactory performance in downstream tasks. The introduction of "multi-modal" in the field of remote sensing research should not be limited to the use of multi-source data, but more importantly to the coding of diverse data and the extracted deep features based on the huge amount of data. Therefore, in this paper, based on an image-text matching model, we establish a multi-modal scene classification model (Fig. 1) for high spatial resolution aerial images which is dominated by image features and text provides facilitation for the representation of image features. The algorithm first employs self-supervised learning of the visual model, to align the expression domain of the image features obtained from training on natural images with that of our particular dataset, which will help to improve the feature extraction effectiveness of the aerial survey images on the visual model. The features generated by the pre-trained image encoding model and the text encoding model will be further aligned and some of the parameters in the image encoder will be iteratively updated during training. A valid classifier is designed at the end of the model to implement the scene classification task. Through experiments, it was found that our algorithm has a significant improvement effect on the task of scene categorization on aerial survey images compared to single visual models. The model presented in the article obtained precision and recall of above 90% on the test dataset, contained in the high spatial resolution aerial survey images dataset we built with 27 categories (Fig. 2). Fig 1. Diagram of the proposed model structure. Blue boxes are associated with the image, green boxes with the text, and red boxes with both image and text. Fig 2. Samples in our high spatial resolution aerial survey images dataset.

Improvememt of Multi-Temporal Vegetation Modeling Using Hybrid Deep Neural Networks of Multispectral Remote Sensing Images.

BI-DIRECTIONAL LSTM MODEL FOR CLASSIFICATION OF VEGETATION FROM SATELLITE TIME SERIES

Object-Based Multi-Temporal and Multi-Source Land Cover Mapping Leveraging Hierarchical Class Relationships

Land Cover Classification of Remote Sensing Imagery with Hybrid Two-Layer Attention Network Architecture

Improving Urban Land Cover/Use Mapping by Integrating A Hybrid Convolutional Neural Network and an Automatic Training Sample Expanding Strategy

Multisensor Land Cover Classification With Sparsely Annotated Data Based on Convolutional Neural Networks and Self-Distillation

Multi-modal learning for geospatial vegetation forecasting

Deep Neural Networks for automatic extraction of features in time series satellite images

Evaluation of Deep Learning Semantic Segmentation for Land Cover Mapping on Multispectral, Hyperspectral and High Spatial Aerial Imagery

Land cover classification from multi-temporal, multi-spectral remotely sensed imagery using patch-based recurrent neural networks

Spatiotemporal Assessment of Satellite Image Time Series for Land Cover Classification Using Deep Learning Techniques: A Case Study of Reunion Island, France

Multisource High-Resolution Remote Sensing Image Vegetation Extraction with Comprehensive Multifeature Perception

Ensemble of Deep Learning-Based Multimodal Remote Sensing Image Classification Model on Unmanned Aerial Vehicle Networks

A Multi-Modal High Spatial Resolution Aerial Imagery Scene Classification Model with Visual Enhancement

EVALUATION OF DIFFERENT PARAMETERS FOR PLANT CLASSIFICATION BY PRE-TRAINED DEEP LEARNING MODELS WITH BIGEARTHNET DATASET

Residual Multi-Attention Classification Network for A Forest Dominated Tropical Landscape Using High-Resolution Remote Sensing Imagery

MARC-Net: Terrain Classification in Parallel Network Architectures Containing Multiple Attention Mechanisms and Multi-Scale Residual Cascades

Advancing crop classification in smallholder agriculture: A multifaceted approach combining frequency-domain image co-registration, transformer-based parcel segmentation, and Bi-LSTM for crop classification

Tree species classification at the pixel-level using deep learning and multispectral time series in an imbalanced context

A Dual-Branch Deep Learning Architecture for Multisensor and Multitemporal Remote Sensing Semantic Segmentation

Intelligent Multi-Level Feature Fusion Using Remote Sensing and CNN Image Classification Algorithm