Abstract:Small satellite constellations provide daily global coverage of the earth's landmass, but image enrichment relies on automating key tasks like change detection or feature searches. For example, to extract text annotations from raw pixels requires two dependent machine learning models, one to analyze the overhead image and the other to generate a descriptive caption. We evaluate seven models on the previously largest benchmark for satellite image captions. We extend the labeled image samples five-fold, then augment, correct and prune the vocabulary to approach a rough min-max (minimum word, maximum description). This outcome compares favorably to previous work with large pre-trained image models but offers a hundred-fold reduction in model size without sacrificing overall accuracy (when measured with log entropy loss). These smaller models provide new deployment opportunities, particularly when pushed to edge processors, on-board satellites, or distributed ground stations. To quantify a caption's descriptiveness, we introduce a novel multi-class confusion or error matrix to score both human-labeled test data and never-labeled images that include bounding box detection but lack full sentence captions. This work suggests future captioning strategies, particularly ones that can enrich the class coverage beyond land use applications and that lessen color-centered and adjacency adjectives ("green", "near", "between", etc.). Many modern language transformers present novel and exploitable models with world knowledge gleaned from training from their vast online corpus. One interesting, but easy example might learn the word association between wind and waves, thus enriching a beach scene with more than just color descriptions that otherwise might be accessed from raw pixels without text annotation.

Do Sentence Transformers Learn Quasi-Geospatial Concepts from General Text?

TransKGQA: Enhanced Knowledge Graph Question Answering With Sentence Transformers

Adapting Sentence Transformers for the Aviation Domain

A Bilingual Generative Transformer for Semantic Sentence Embedding

Formal Semantic Geometry over Transformer-based Variational AutoEncoder

MetaQA: Enhancing human-centered data search using Generative Pre-trained Transformer (GPT) language model and artificial intelligence

A Brief Overview of Universal Sentence Representation Methods: A Linguistic View

Inducing Systematicity in Transformers by Attending to Structurally Quantized Embeddings

Language Query-Based Transformer With Multiscale Cross-Modal Alignment for Visual Grounding on Remote Sensing Images

Syntax-informed Question Answering with Heterogeneous Graph Transformer

Towards Understanding and Answering Multi-Sentence Recommendation Questions on Tourism

Transcormer: Transformer for Sentence Scoring with Sliding Language Modeling

Segatron: Segment-Aware Transformer for Language Modeling and Understanding

A new approach for fine-tuning sentence transformers for intent classification and out-of-scope detection tasks

Diversifying Query: Region-Guided Transformer for Temporal Sentence Grounding

Vision Transformers with Natural Language Semantics

Discoverability in Satellite Imagery: A Good Sentence is Worth a Thousand Pictures

Local-to-Global Structure-Aware Transformer for Question Answering over Structured Knowledge.

Can Transformers Capture Spatial Relations between Objects?

Tracking linguistic information in transformer-based sentence embeddings through targeted sparsification

Semantic Computation in Geography Question Answering