Abstract:The emergence of large-scale large language models, with GPT-4 as a prominent example, has significantly propelled the rapid advancement of artificial general intelligence and sparked the revolution of Artificial Intelligence 2.0. In the realm of remote sensing (RS), there is a growing interest in developing large vision language models (VLMs) specifically tailored for data analysis in this domain. However, current research predominantly revolves around visual recognition tasks, lacking comprehensive, large-scale image-text datasets that are aligned and suitable for training large VLMs, which poses significant challenges to effectively training such models for RS applications. In computer vision, recent research has demonstrated that fine-tuning large vision language models on small-scale, high-quality datasets can yield impressive performance in visual and language understanding. These results are comparable to state-of-the-art VLMs trained from scratch on massive amounts of data, such as GPT-4. Inspired by this captivating idea, in this work, we build a high-quality Remote Sensing Image Captioning dataset (RSICap) that facilitates the development of large VLMs in the RS field. Unlike previous RS datasets that either employ model-generated captions or short descriptions, RSICap comprises 2,585 human-annotated captions with rich and high-quality information. This dataset offers detailed descriptions for each image, encompassing scene descriptions (e.g., residential area, airport, or farmland) as well as object information (e.g., color, shape, quantity, absolute position, etc). To facilitate the evaluation of VLMs in the field of RS, we also provide a benchmark evaluation dataset called RSIEval. This dataset consists of human-annotated captions and visual question-answer pairs, allowing for a comprehensive assessment of VLMs in the context of RS.

On Creating Benchmark Dataset for Aerial Image Interpretation: Reviews, Guidances, and Million-AID

Benchmarking Large-Scale Multi-View 3D Reconstruction Using Realistic Synthetic Images

AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification

VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding

PatternNet: A benchmark dataset for performance evaluation of remote sensing image retrieval

Two novel benchmark datasets from ArcGIS and bing world imagery for remote sensing image retrieval

RSGPT: A Remote Sensing Vision Language Model and Benchmark

AI applications in forest monitoring need remote sensing benchmark datasets

RSI-CB: A Large-Scale Remote Sensing Image Classification Benchmark Using Crowdsourced Data

iSAID: A Large-scale Dataset for Instance Segmentation in Aerial Images

RSI-CB: A Large Scale Remote Sensing Image Classification Benchmark via Crowdsource Data

Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark

Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges

RSSOD-Bench: A large-scale benchmark dataset for Salient Object Detection in Optical Remote Sensing Imagery

An Aerial Photogrammetry Benchmark Dataset for Point Cloud Segmentation and Style Translation

A Review of Deep Learning-Based Remote Sensing Image Caption: Methods, Models, Comparisons and Future Directions

Image retrieval from remote sensing big data: A survey

ReCon1M:A Large-scale Benchmark Dataset for Relation Comprehension in Remote Sensing Imagery

AI Competitions and Benchmarks: Dataset Development

Csrs-Siat: A Benchmark Remote Sensing Dataset to Semantic-Enabled and Cross-Scales Scene Recognition