Abstract:Recently, deep learning (DL) presents a promising performance in the joint classification of multimodal remote sensing (RS) data. However, most of the approaches adopt a supervised learning (SL) manner, where the discrimination capability is limited by the paucity of labeled samples. Though some attempts have been made to develop semi-supervised methods, they prefer to select the highly confident predictions as pseudo-ground truth and discard those unreliable ones. Actually, unreliable samples can also provide useful information, e.g., indicating the categories to which samples may belong and definitely not belong. Focused on this, a novel uncertainty-aware contrastive learning (UACL) method is proposed. Here, label uncertainty analysis (UA) based on multilevel probability estimation is first conducted to separate reliable and unreliable samples, which are then processed with a designed hybrid ("hard" or "soft") contrastive learning (CL) strategy. For reliable samples, the "hard" CL pushes the network to learn features that will minimize the intra-class distance while maximizing the inter-class distance, according to the pseudo-labels. For unreliable samples, the "soft" CL aims to learn the similarity and difference among samples, where the predicted class probabilities are queried to estimate a soft mask for an adaptive feature similarity measurement. Moreover, a multimodal spectral-spatial joint feature representation pipeline of triple branches, i.e., one spectral branch for hyperspectral images (HSIs) and two spatial branches for multimodal data, is also introduced. By jointly learning from both labeled and unlabeled samples, more discriminative spectral-spatial feature representation will lead to a further boost in classification performance. Extensive experiments on four well-known multimodal datasets prove the effectiveness of the proposed semi-supervised classification method. Codes are available at https://github.com/Ding-Kexin/UACL.

UCoL: Unsupervised Learning of Discriminative Facial Representations via Uncertainty-Aware Contrast

Uncertainty-Aware Contrastive Learning for Semi-Supervised Classification of Multimodal Remote Sensing Images

Contrastive Learning with Hallucinating Data for Long-Tailed Face Recognition.

Weakly-Supervised Text-driven Contrastive Learning for Facial Behavior Understanding

Pose-disentangled Contrastive Learning for Self-supervised Facial Representation

Facial Expression Recognition with Contrastive Learning and Uncertainty-Guided Relabeling

Unsupervised Facial Expression Representation Learning with Contrastive Local Warping

Pursuing Knowledge Consistency: Supervised Hierarchical Contrastive Learning for Facial Action Unit Recognition

Self-Supervised Visual Representations Learning by Contrastive Mask Prediction

Occluded Facial Expression Recognition Using Self-supervised Learning

Uncertainty-oriented Order Learning for Facial Beauty Prediction

DeepfakeUCL: Deepfake Detection via Unsupervised Contrastive Learning

CO2: Consistent Contrast for Unsupervised Visual Representation Learning

Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition

Self-supervised extracted contrast network for facial expression recognition

Unsupervised Voice-Face Representation Learning by Cross-Modal Prototype Contrast

P4Contrast: Contrastive Learning with Pairs of Point-Pixel Pairs for RGB-D Scene Understanding

Learning the Unlearned: Mitigating Feature Suppression in Contrastive Learning

Unified Contrastive Learning in Image-Text-Label Space

CoReFace: Sample-Guided Contrastive Regularization for Deep Face Recognition

Learning Contrastive Feature Representations for Facial Action Unit Detection