EarthNets: Empowering AI in Earth Observation

Zhitong Xiong,Fahong Zhang,Yi Wang,Yilei Shi,Xiao Xiang Zhu

2024-04-03

Abstract:Earth observation (EO), aiming at monitoring the state of planet Earth using remote sensing data, is critical for improving our daily lives and living environment. With a growing number of satellites in orbit, an increasing number of datasets with diverse sensors and research domains are being published to facilitate the research of the remote sensing community. This paper presents a comprehensive review of more than 500 publicly published datasets, including research domains like agriculture, land use and land cover, disaster monitoring, scene understanding, vision-language models, foundation models, climate change, and weather forecasting. We systematically analyze these EO datasets from four aspects: volume, resolution distributions, research domains, and the correlation between datasets. Based on the dataset attributes, we propose to measure, rank, and select datasets to build a new benchmark for model evaluation. Furthermore, a new platform for EO, termed EarthNets, is released to achieve a fair and consistent evaluation of deep learning methods on remote sensing data. EarthNets supports standard dataset libraries and cutting-edge deep learning models to bridge the gap between the remote sensing and machine learning communities. Based on this platform, extensive deep-learning methods are evaluated on the new benchmark. The insightful results are beneficial to future research. The platform and dataset collections are publicly available at

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper mainly focuses on a comprehensive review, systematic analysis, and benchmark construction of remote sensing datasets in the field of Earth Observation (EO). With the increasing number of satellite launches, a large amount of remote sensing data is being generated, which has wide applications in various fields such as agriculture, land cover, disaster monitoring, and scene understanding. Although deep learning technologies have shown excellent performance in handling large-scale data, there is still a lack of comprehensive evaluation and comparison of remote sensing datasets.

The paper first conducts a detailed review of over 500 publicly available remote sensing datasets and categorizes them into image classification, object detection, semantic segmentation, change detection, visual language understanding, and base models based on research tasks. The authors analyze the volume, resolution distribution, research areas, and relationships between these datasets. They propose a measurement, ranking, and selection method based on dataset attributes to create a new benchmark for evaluating model performance.

In addition, the paper introduces a new platform called "EarthNets", which aims to facilitate collaboration between the remote sensing and machine learning communities by providing a standardized dataset repository and the latest deep learning models for fair and consistent algorithm evaluation. Through this platform, researchers can easily find remote sensing data suitable for specific applications, accelerating the development and comparison of methods.

In conclusion, this paper aims to address the lack of comprehensive review, systematic analysis, unified benchmark, and open platform for remote sensing datasets. By providing a comprehensive dataset analysis, new benchmarks, and an open-source platform, it promotes interdisciplinary research progress in remote sensing and artificial intelligence.