Abstract:Recently, deep learning (DL) presents a promising performance in the joint classification of multimodal remote sensing (RS) data. However, most of the approaches adopt a supervised learning (SL) manner, where the discrimination capability is limited by the paucity of labeled samples. Though some attempts have been made to develop semi-supervised methods, they prefer to select the highly confident predictions as pseudo-ground truth and discard those unreliable ones. Actually, unreliable samples can also provide useful information, e.g., indicating the categories to which samples may belong and definitely not belong. Focused on this, a novel uncertainty-aware contrastive learning (UACL) method is proposed. Here, label uncertainty analysis (UA) based on multilevel probability estimation is first conducted to separate reliable and unreliable samples, which are then processed with a designed hybrid ("hard" or "soft") contrastive learning (CL) strategy. For reliable samples, the "hard" CL pushes the network to learn features that will minimize the intra-class distance while maximizing the inter-class distance, according to the pseudo-labels. For unreliable samples, the "soft" CL aims to learn the similarity and difference among samples, where the predicted class probabilities are queried to estimate a soft mask for an adaptive feature similarity measurement. Moreover, a multimodal spectral-spatial joint feature representation pipeline of triple branches, i.e., one spectral branch for hyperspectral images (HSIs) and two spatial branches for multimodal data, is also introduced. By jointly learning from both labeled and unlabeled samples, more discriminative spectral-spatial feature representation will lead to a further boost in classification performance. Extensive experiments on four well-known multimodal datasets prove the effectiveness of the proposed semi-supervised classification method. Codes are available at https://github.com/Ding-Kexin/UACL.

Occluded Scene Classification via Cascade Supervised Contrastive Learning

A lightweight and stochastic depth residual attention network for remote sensing scene classification

Omnisupervised Omnidirectional Semantic Segmentation

Saliency Guided Contrastive Learning on Scene Images

Dual Contrastive Network for Few-Shot Remote Sensing Image Scene Classification

Spatial and Semantic Consistency Contrastive Learning for Self-Supervised Semantic Segmentation of Remote Sensing Images

Cascaded panoptic segmentation method for high resolution remote sensing image

Compositional Convolutional Neural Networks: A Deep Architecture with Innate Robustness to Partial Occlusion

Combining Compositional Models and Deep Networks For Robust Object Classification under Occlusion

Few-Shot Remote Sensing Scene Classification With Spatial Affinity Attention and Class Surrogate-Based Supervised Contrastive Learning

Contrastive Learning for Urban Land Cover Classification With Multimodal Siamese Network

Contrast-Oriented Deep Neural Networks for Salient Object Detection

TDAPNet: Prototype Network with Recurrent Top-Down Attention for Robust Object Classification under Partial Occlusion

Supervised contrastive learning with multi-scale interaction and integrity learning for salient object detection

When Self-Supervised Learning Meets Scene Classification: Remote Sensing Scene Classification Based on a Multitask Learning Framework

Occluded Facial Expression Recognition with Step-Wise Assistance from Unpaired Non-Occluded Images.

Multimodal Supervised Contrastive Learning in Remote Sensing Downstream Tasks

High Quality Object Detection for Multiresolution Remote Sensing Imagery Using Cascaded Multi-Stage Detectors.

Hybrid Task Cascade for Instance Segmentation

Uncertainty-Aware Contrastive Learning for Semi-Supervised Classification of Multimodal Remote Sensing Images