Abstract:Remote sensing image captioning involves remote sensing objects and their spatial relationships. However, it is still difficult to determine the spatial extent of a remote sensing object and the size of a sample patch. If the patch size is too large, it will include too many remote sensing objects and their complex spatial relationships. This will increase the computational burden of the image captioning network and reduce its precision. If the patch size is too small, it often fails to provide enough environmental and contextual information, which makes the remote sensing object difficult to describe. To address this problem, we propose a multi-scale semantic long short-term memory network (MS-LSTM). The remote sensing images are paired into image patches with different spatial scales. First, the large-scale patches have larger sizes. We use a Visual Geometry Group (VGG) network to extract the features from the large-scale patches and input them into the improved MS-LSTM network as the semantic information, which provides a larger receptive field and more contextual semantic information for small-scale image caption so as to play the role of global perspective, thereby enabling the accurate identification of small-scale samples with the same features. Second, a small-scale patch is used to highlight remote sensing objects and simplify their spatial relations. In addition, the multi-receptive field provides perspectives from local to global. The experimental results demonstrated that compared with the original long short-term memory network (LSTM), the MS-LSTM's Bilingual Evaluation Understudy (BLEU) has been increased by 5.6% to 0.859, thereby reflecting that the MS-LSTM has a more comprehensive receptive field, which provides more abundant semantic information and enhances the remote sensing image captions.

Deep Semantic Understanding of High Resolution Remote Sensing Image

Exploring Multi-Level Attention and Semantic Relationship for Remote Sensing Image Captioning

Semantic Descriptions of High-Resolution Remote Sensing Images

Semantic Relation Model and Dataset for Remote Sensing Scene Understanding

NATURAL LANGUAGE DESCRIPTION OF REMOTE SENSING IMAGES BASED ON DEEP LEARNING

Regenerating Image Caption with High-Level Semantics.

What Do We Learn by Semantic Scene Understanding for Remote Sensing Imagery in CNN Framework?

Transformer and CNN Hybrid Deep Neural Network for Semantic Segmentation of Very-High-Resolution Remote Sensing Imagery

Densely Based Multi-Scale and Multi-Modal Fully Convolutional Networks for High-Resolution Remote-Sensing Image Semantic Segmentation

High-Resolution Aerial Images Semantic Segmentation Using Deep Fully Convolutional Network with Channel Attention Mechanism

Multi-Scale Remote Sensing Semantic Analysis Based On A Global Perspective

High-Resolution Remote Sensing Image Captioning Based on Structured Attention

Remote Sensing Image Captioning Based on Multi-Level Feature Extraction and Adaptive Attention

Semantic Segmentation for High-Resolution Aerial Imagery Using Multi-Skip Network and Markov Random Fields

Mining Deep Semantic Representations for Scene Classification of High-Resolution Remote Sensing Imagery

A Semantic-Driven Image Scene Fine-Grained Enhancement Recognition

Learning Consensus-Aware Semantic Knowledge for Remote Sensing Image Captioning

Hierarchical Self-Attention Embedded Neural Network With Dense Connection for Remote-Sensing Image Semantic Segmentation

Image Caption Model of Double LSTM with Scene Factors

Remote Sensing Image Caption Generation Via Transformer and Reinforcement Learning.

Research on semantic segmentation algorithm of multispectral remote sensing image based on deep learning