Abstract:Remote sensing projects typically generate large amounts of imagery that can be used to train powerful deep neural networks. However, the amount of labeled images is often small, as remote sensing applications generally require expert labelers. Thus, semi-supervised learning (SSL), i.e., learning with a small pool of labeled and a larger pool of unlabeled data, is particularly useful in this domain. Current SSL approaches generate pseudo-labels from model predictions for unlabeled samples. As the quality of these pseudo-labels is crucial for performance, utilizing additional information to improve pseudo-label quality yields a promising direction. For remote sensing images, geolocation and recording time are generally available and provide a valuable source of information as semantic concepts, such as land cover, are highly dependent on spatiotemporal context, e.g., due to seasonal effects and vegetation zones. In this paper, we propose to exploit spatiotemporal metainformation in SSL to improve the quality of pseudo-labels and, therefore, the final model performance. We show that directly adding the available metadata to the input of the predictor at test time degenerates the prediction quality for metadata outside the spatiotemporal distribution of the training set. Thus, we propose a teacher-student SSL framework where only the teacher network uses metainformation to improve the quality of pseudo-labels on the training set. Correspondingly, our student network benefits from the improved pseudo-labels but does not receive metadata as input, making it invariant to spatiotemporal shifts at test time. Furthermore, we propose methods for encoding and injecting spatiotemporal information into the model and introduce a novel distillation mechanism to enhance the knowledge transfer between teacher and student. Our framework dubbed Spatiotemporal SSL can be easily combined with several stat...

Lightweight Contenders: Navigating Semi-Supervised Text Mining through Peer Collaboration and Self Transcendence

DisCo: Remedying Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning

DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning

DisCo: Distilled Student Models Co-training for Semi-supervised Text Mining

FLiText: A Faster and Lighter Semi-Supervised Text Classification with Convolution Networks

Meta-Semi: A Meta-learning Approach for Semi-supervised Learning.

SemiNLL: A Framework of Noisy-Label Learning by Semi-Supervised Learning

Rethinking Semi-supervised Learning with Language Models

FlexSSL : A Generic and Efficient Framework for Semi-Supervised Learning

SemiReward: A General Reward Model for Semi-supervised Learning

When Semi-Supervised Learning Meets Transfer Learning: Training Strategies, Models and Datasets.

Neural collapse inspired semi-supervised learning with fixed classifier

Context Matters: Leveraging Spatiotemporal Metadata for Semi-Supervised Learning on Remote Sensing Images

Learning to Infer from Unlabeled Data: A Semi-supervised Learning Approach for Robust Natural Language Inference

Label Diagnosis Through Self Tuning for Web Image Search

Semi-Supervised Learning via Weight-aware Distillation under Class Distribution Mismatch

Generalized Semi-Supervised Learning via Self-Supervised Feature Adaptation

Robust Semi-Supervised Learning when Not All Classes have Labels

Heterogeneous Student Knowledge Distillation From BERT Using a Lightweight Ensemble Framework

Robust Deep Semi-Supervised Learning: A Brief Introduction

DP-SSL: Towards Robust Semi-supervised Learning with A Few Labeled Samples