Abstract:This work proposes a hybrid unsupervised and supervised learning method to pre-train models applied in Earth observation downstream tasks when only a handful of labels denoting very general semantic concepts are available. We combine a contrastive approach to pre-train models with a pixel-wise regression pre-text task to predict coarse elevation maps, which are commonly available worldwide. We hypothesize that this will allow the model to pre-learn useful representations, as there is generally some correlation between elevation maps and targets in many remote sensing tasks. We assess the performance of our approach on a binary semantic segmentation task and a binary image classification task, both derived from a dataset created for the northwest of Colombia. In both cases, we pre-train our models with 39k unlabeled images, fine-tune them on the downstream tasks with only 80 labeled images, and evaluate them with 2944 labeled images. Our experiments show that our methods, GLCNet+Elevation for segmentation, and SimCLR+Elevation for classification, outperform their counterparts without the pixel-wise regression pre-text task, namely SimCLR and GLCNet, in terms of macro-average F1 Score and Mean Intersection over Union (MIoU). Our study not only encourages the development of pre-training methods that leverage readily available geographical information, such as elevation data, to enhance the performance of self-supervised methods when applied to Earth observation tasks, but also promotes the use of datasets with high-level semantic labels, which are more likely to be updated frequently. Project code can be found in this link \href{<a class="link-external link-https" href="https://github.com/omarcastano/Elevation-Aware-SSL" rel="external noopener nofollow">this https URL</a>}{<a class="link-external link-https" href="https://github.com/omarcastano/Elevation-Aware-SSL" rel="external noopener nofollow">this https URL</a>}.

Semi-Supervised Semantic Segmentation in Earth Observation: The MiniFrance Suite, Dataset Analysis and Multi-task Network Study

Semi-Supervised Learning for Visual Bird's Eye View Semantic Segmentation

In Defense Of Multi-Source Omni-Supervised Efficient Convnet For Robust Semantic Segmentation In Heterogeneous Unseen Domains

Deep learning for semantic segmentation of remote sensing images with rich spectral content

MULTISENGE: A MULTIMODAL AND MULTITEMPORAL BENCHMARK DATASET FOR LAND USE/LAND COVER REMOTE SENSING APPLICATIONS

Synthetic Data for Sentinel-2 Semantic Segmentation

Semi-Supervised Adversarial Semantic Segmentation Network Using Transformer and Multiscale Convolution for High-Resolution Remote Sensing Imagery

Category-sensitive Semi-Supervised Semantic Segmentation Framework for Land-Use/land-cover Mapping with Optical Remote Sensing Images

Segmentation of VHR EO Images using Unsupervised Learning

A Dual Network for Super-Resolution and Semantic Segmentation of Sentinel-2 Imagery

S$^{5}$Mars: Semi-Supervised Learning for Mars Semantic Segmentation

Semantic Segmentation of Remote Sensing Images With Self-Supervised Multitask Representation Learning

Simple and Efficient: A Semisupervised Learning Framework for Remote Sensing Image Semantic Segmentation

Semi-Supervised Remote Sensing Image Semantic Segmentation Method Based on Deep Learning

S^5Mars: Semi-Supervised Learning for Mars Semantic Segmentation

Semantic Segmentation of Vegetation in Remote Sensing Imagery Using Deep Learning

Semi-Automated Segmentation of Geoscientific Data Using Superpixels

DiverseNet: Decision Diversified Semi-supervised Semantic Segmentation Networks for Remote Sensing Imagery

Enhancing Self-Supervised Learning for Remote Sensing with Elevation Data: A Case Study with Scarce And High Level Semantic Labels

Semantic Stereo for Incidental Satellite Images

S5Mars: Semi-Supervised Learning for Mars Semantic Segmentation