SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understanding

Favyen Bastani,Piper Wolters,Ritwik Gupta,Joe Ferdinando,Aniruddha Kembhavi

2023-08-21

Abstract:Remote sensing images are useful for a wide variety of planet monitoring applications, from tracking deforestation to tackling illegal fishing. The Earth is extremely diverse -- the amount of potential tasks in remote sensing images is massive, and the sizes of features range from several kilometers to just tens of centimeters. However, creating generalizable computer vision methods is a challenge in part due to the lack of a large-scale dataset that captures these diverse features for many tasks. In this paper, we present SatlasPretrain, a remote sensing dataset that is large in both breadth and scale, combining Sentinel-2 and NAIP images with 302M labels under 137 categories and seven label types. We evaluate eight baselines and a proposed method on SatlasPretrain, and find that there is substantial room for improvement in addressing research challenges specific to remote sensing, including processing image time series that consist of images from very different types of sensors, and taking advantage of long-range spatial context. Moreover, we find that pre-training on SatlasPretrain substantially improves performance on downstream tasks, increasing average accuracy by 18% over ImageNet and 6% over the next best baseline. The dataset, pre-trained model weights, and code are available at <a class="link-external link-https" href="https://satlas-pretrain.allen.ai/" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to address several key challenges in automated remote sensing image understanding methods. Specifically: 1. **Dataset Scale and Diversity**: Existing remote sensing datasets are relatively small and have limited categories, which cannot fully leverage the advantages of modern computer vision architectures (such as ViT and CLIP). This paper proposes a large-scale, multi-task remote sensing dataset called SATLAS PRETRAIN, which includes 137 categories and 7 types of labels, aiming to overcome this limitation. 2. **Pre-training Effectiveness**: The study found that pre-training on SATLAS PRETRAIN can significantly improve the performance of downstream tasks, with an 18% improvement compared to ImageNet and 6% higher than other baselines. 3. **Unified Model Design**: To address the issue that existing models cannot handle all types of labels, the authors developed a unified model named SATLAS NET, which supports the prediction of all seven types of labels and further enhances performance through joint training. In summary, this paper is dedicated to advancing the field of automated remote sensing image understanding by creating a large-scale and diverse remote sensing dataset and a corresponding unified model.

SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understanding

Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery

SATIN: A Multi-Task Metadataset for Classifying Satellite Imagery using Vision-Language Models

Advancing Remote Sensing: A Unified Deep Learning Approach with Pretrained and Custom Architectures for High-Precision Classification

SSL4EO-S12: A Large-Scale Multi-Modal, Multi-Temporal Dataset for Self-Supervised Learning in Earth Observation

Large Scale Masked Autoencoding for Reducing Label Requirements on SAR Data

WildSAT: Learning Satellite Image Representations from Wildlife Observations

Self-Supervised Pretraining on Satellite Imagery: a Case Study on Label-Efficient Vehicle Detection

SSL4EO-S12: A large-scale multimodal, multitemporal dataset for self-supervised learning in Earth observation [Software and Data Sets]

Learning to "Segment Anything" in Thermal Infrared Images through Knowledge Distillation with a Large Scale Dataset SATIR

SaRNet: A Dataset for Deep Learning Assisted Search and Rescue with Satellite Imagery

Revisiting pre-trained remote sensing model benchmarks: resizing and normalization matters

MLRSNet: A multi-label high spatial resolution remote sensing dataset for semantic scene understanding

SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery

SatVision-TOA: A Geospatial Foundation Model for Coarse-Resolution All-Sky Remote Sensing Imagery

SAR-HUB: Pre-Training, Fine-Tuning, and Explaining

Self-supervised Audiovisual Representation Learning for Remote Sensing Data

An Empirical Study of Remote Sensing Pretraining

Geographical Knowledge-driven Representation Learning for Remote Sensing Images

Enhancing Self-Supervised Learning for Remote Sensing with Elevation Data: A Case Study with Scarce And High Level Semantic Labels

Consecutive Pre-Training: A Knowledge Transfer Learning Strategy with Relevant Unlabeled Data for Remote Sensing Domain