Abstract:Self-supervised learning (SSL) has gained wide-spread attention in the remote sensing (RS) and Earth observation (EO) communities owing to its ability to learn task-agnostic representations without human-annotated labels. Nevertheless, most existing RS SSL methods are limited to learning either global semantic separable or local spatial perceptible representations. We argue that this learning strategy is suboptimal in the realm of RS since the required representations for different RS downstream tasks are often varied and complex. In this study, we proposed a unified SSL framework that is better suited for RS image representation learning. The proposed SSL framework, contrastive mask image distillation (CMID), is capable of learning representations with both global semantic separability and local spatial perceptibility by combining contrastive learning (CL) with masked image modeling (MIM) in a self-distillation way. Furthermore, our CMID learning framework is architecture-agnostic, which is compatible with both convolutional neural networks (CNNs) and vision transformers (ViTs), allowing CMID to be easily adapted to a variety of deep learning (DL) applications for RS understanding. Comprehensive experiments have been carried out on four downstream tasks (i.e., scene classification, semantic segmentation, object detection, and change detection) and the results show that models pretrained using CMID achieve a better performance than other state-of-the-art SSL methods on multiple downstream tasks. The code and pretrained models will be made available at https://github.com/NJU-LHRS/official-CMIDhttps://github.com/NJU-LHRS/official-CMID to facilitate SSL research and speed up the development of RS images DL applications.

Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problems

On Learning Discriminative Features from Synthesized Data for Self-Supervised Fine-Grained Visual Recognition

PP-SSL : Priority-Perception Self-Supervised Learning for Fine-Grained Recognition

Learning Where to Learn in Cross-View Self-Supervised Learning

On the Discriminability of Self-Supervised Representation Learning

Align Yourself: Self-supervised Pre-training for Fine-grained Recognition via Saliency Alignment.

Occam's Razor for Self Supervised Learning: What is Sufficient to Learn Good Representations?

Measuring Self-Supervised Representation Quality for Downstream Classification using Discriminative Features

Self-Supervised Representation Learning with Meta Comprehensive Regularization

Self-supervised Video Representation Learning via Capturing Semantic Changes Indicated by Saccades

Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition

Deciphering the Projection Head: Representation Evaluation Self-supervised Learning

HIRL: A General Framework for Hierarchical Image Representation Learning

The Common Stability Mechanism behind most Self-Supervised Learning Approaches

Understanding and Improving the Role of Projection Head in Self-Supervised Learning

A generic self-supervised learning (SSL) framework for representation learning from spectra-spatial feature of unlabeled remote sensing imagery

Weak Augmentation Guided Relational Self-Supervised Learning

Rethinking Evaluation Protocols of Visual Representations Learned via Self-supervised Learning

CMID: A Unified Self-Supervised Learning Framework for Remote Sensing Image Understanding

Siamese Image Modeling for Self-Supervised Vision Representation Learning

Boosting Fine-Grained Sketch-Based Image Retrieval with Self-Supervised Learning