Abstract:Recently, supervised deep learning has achieved a great success in remote sensing image (RSI) semantic segmentation. However, supervised learning for semantic segmentation requires a large number of labeled samples, which is difficult to obtain in the field of remote sensing. A new learning paradigm, self-supervised learning (SSL), can be used to solve such problems by pretraining a general model with a large number of unlabeled images and then fine-tuning it on a downstream task with very few labeled samples. Contrastive learning is a typical method of SSL that can learn general invariant features. However, most existing contrastive learning methods are designed for classification tasks to obtain an image-level representation, which may be suboptimal for semantic segmentation tasks requiring pixel-level discrimination. Therefore, we propose a global style and local matching contrastive learning network (GLCNet) for RSI semantic segmentation. Specifically, first, the global style contrastive learning module is used to better learn an image-level representation, as we consider that style features can better represent the overall image features. Next, the local features matching the contrastive learning module is designed to learn the representations of local regions, which is beneficial for semantic segmentation. We evaluate four RSI semantic segmentation datasets, and the experimental results show that our method mostly outperforms the state-of-the-art self-supervised methods and the ImageNet pretraining method. Specifically, with 1% annotation from the original dataset, our approach improves Kappa by 6% on the International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam dataset relative to the existing baseline. Moreover, our method outperforms supervised learning methods when there are some differences between the datasets of upstream tasks and downstream tasks. Our study promotes the development of SSL in the field of RSI semantic segmentation. Since SSL could directly learn the essential characteristics of data from unlabeled data, which is easy to obtain in the remote sensing field, this may be of great significance for tasks such as global mapping. The source code is available at https://github.com/GeoX-Lab/G-RSIM .

Self-Supervised Learning in Remote Sensing: A review

Self-Supervised Learning on Small In-Domain Datasets Can Overcome Supervised Learning in Remote Sensing

Self-supervised Learning on Graphs: Contrastive, Generative,or Predictive

Remote Sensing Image Scene Classification with Self-Supervised Paradigm under Limited Labeled Samples

Self-supervised Learning for Geospatial AI: A Survey

Progress and Thinking on Self-Supervised Learning Methods in Computer Vision: A Review

A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends

A generic self-supervised learning (SSL) framework for representation learning from spectra-spatial feature of unlabeled remote sensing imagery

In-Domain Self-Supervised Learning Improves Remote Sensing Image Scene Classification

A Survey of Self-Supervised Learning from Multiple Perspectives: Algorithms, Theory, Applications and Future Trends

Remote Sensing Images Semantic Segmentation with General Remote Sensing Vision Model via a Self-Supervised Contrastive Learning Method

Self-Supervised Learning and Its Applications in Ophthalmology

Open Self-Supervised Features for Remote-Sensing Image Scene Classification Using Very Few Samples

Self-Supervised Remote Sensing Feature Learning: Learning Paradigms, Challenges, and Future Works

Deep Learning-based Semantic Segmentation of Remote Sensing Images: A Survey

Remote sensing image intelligent interpretation:from supervised learning to self-supervised learning

Self-Supervised Learning for Point Clouds Data: A Survey

SSL4EO-S12: A Large-Scale Multi-Modal, Multi-Temporal Dataset for Self-Supervised Learning in Earth Observation

SSL4EO-S12: A large-scale multimodal, multitemporal dataset for self-supervised learning in Earth observation [Software and Data Sets]

Global and Local Contrastive Self-Supervised Learning for Semantic Segmentation of HR Remote Sensing Images