Abstract:Remote sensing projects typically generate large amounts of imagery that can be used to train powerful deep neural networks. However, the amount of labeled images is often small, as remote sensing applications generally require expert labelers. Thus, semi-supervised learning (SSL), i.e., learning with a small pool of labeled and a larger pool of unlabeled data, is particularly useful in this domain. Current SSL approaches generate pseudo-labels from model predictions for unlabeled samples. As the quality of these pseudo-labels is crucial for performance, utilizing additional information to improve pseudo-label quality yields a promising direction. For remote sensing images, geolocation and recording time are generally available and provide a valuable source of information as semantic concepts, such as land cover, are highly dependent on spatiotemporal context, e.g., due to seasonal effects and vegetation zones. In this paper, we propose to exploit spatiotemporal metainformation in SSL to improve the quality of pseudo-labels and, therefore, the final model performance. We show that directly adding the available metadata to the input of the predictor at test time degenerates the prediction quality for metadata outside the spatiotemporal distribution of the training set. Thus, we propose a teacher-student SSL framework where only the teacher network uses metainformation to improve the quality of pseudo-labels on the training set. Correspondingly, our student network benefits from the improved pseudo-labels but does not receive metadata as input, making it invariant to spatiotemporal shifts at test time. Furthermore, we propose methods for encoding and injecting spatiotemporal information into the model and introduce a novel distillation mechanism to enhance the knowledge transfer between teacher and student. Our framework dubbed Spatiotemporal SSL can be easily combined with several stat...

Semantic context learning with large-scale weakly-labeled image set.

Semantic Image Retrieval Based on Multiple-Instance Learning

Semantic Image Segmentation Based on Spatial Context Relations

A Semantic Context Model For Automatic Image Annotation

Weakly Supervised Semantic Segmentation for Social Images

Semantic Segmentation Based On Stacked Discriminative Autoencoders And Context-Constrained Weakly Supervised Learning

Webly-supervised semantic segmentation via curriculum learning

Weakly Supervised Semantic Segmentation with a Multiscale Model

Large-Scale Sparse Learning from Noisy Tags for Semantic Segmentation.

Fast Semantic Diffusion for Large-Scale Context-Based Image and Video Annotation

Weakly Supervised Semantic Segmentation Based on Web Image Co-segmentation

Weakly Supervised Semantic Segmentation for Large-Scale Point Cloud

Automatic Image Annotation with Weakly Labeled Dataset

Latent Visual Context Learning for Web Image Applications

Learning Context-aware Classifier for Semantic Segmentation

Weakly Supervised Learning of Semantic Correspondence Through Cascaded Online Correspondence Refinement

Weaklier Supervised Semantic Segmentation with Only One Image Level Annotation Per Category.

Labeling images by integrating sparse multiple distance learning and semantic context modeling

Image Region Labeling by Exploring Contextual Information of Visual Spatial and Semantic Concepts

Weakly Supervised Semantic Segmentation Based on Co-segmentation.

Context Matters: Leveraging Spatiotemporal Metadata for Semi-Supervised Learning on Remote Sensing Images