Abstract:Recently, deep learning (DL) presents a promising performance in the joint classification of multimodal remote sensing (RS) data. However, most of the approaches adopt a supervised learning (SL) manner, where the discrimination capability is limited by the paucity of labeled samples. Though some attempts have been made to develop semi-supervised methods, they prefer to select the highly confident predictions as pseudo-ground truth and discard those unreliable ones. Actually, unreliable samples can also provide useful information, e.g., indicating the categories to which samples may belong and definitely not belong. Focused on this, a novel uncertainty-aware contrastive learning (UACL) method is proposed. Here, label uncertainty analysis (UA) based on multilevel probability estimation is first conducted to separate reliable and unreliable samples, which are then processed with a designed hybrid ("hard" or "soft") contrastive learning (CL) strategy. For reliable samples, the "hard" CL pushes the network to learn features that will minimize the intra-class distance while maximizing the inter-class distance, according to the pseudo-labels. For unreliable samples, the "soft" CL aims to learn the similarity and difference among samples, where the predicted class probabilities are queried to estimate a soft mask for an adaptive feature similarity measurement. Moreover, a multimodal spectral-spatial joint feature representation pipeline of triple branches, i.e., one spectral branch for hyperspectral images (HSIs) and two spatial branches for multimodal data, is also introduced. By jointly learning from both labeled and unlabeled samples, more discriminative spectral-spatial feature representation will lead to a further boost in classification performance. Extensive experiments on four well-known multimodal datasets prove the effectiveness of the proposed semi-supervised classification method. Codes are available at https://github.com/Ding-Kexin/UACL.

Beyond Cats and Dogs: Semi-supervised Classification of fuzzy labels with overclustering

Fuzzy Overclustering: Semi-Supervised Classification of Fuzzy Labels with Overclustering and Inverse Cross-Entropy

CSAL: Self-adaptive Labeling based Clustering Integrating Supervised Learning on Unlabeled Data

Mejigclu: more effective jigsaw clustering for unsupervised visual representation learning

Fuzzy Positive Learning for Semi-Supervised Semantic Segmentation

Bridge the gap between supervised and unsupervised learning for fine-grained classification

Deep Categorization with Semi-Supervised Self-Organizing Maps

Clustering Algorithms and RAG Enhancing Semi-Supervised Text Classification with Large LLMs

Image Clustering Algorithm Based on Self-Supervised Pretrained Models and Latent Feature Distribution Optimization

Uncertainty-Aware Contrastive Learning for Semi-Supervised Classification of Multimodal Remote Sensing Images

Graph-Based Semi-Supervised Deep Image Clustering With Adaptive Adjacency Matrix

PointCluster: Deep Clustering of 3-D Point Clouds With Semantic Pseudo-Labeling

A Lightweight Clustering Framework for Unsupervised Semantic Segmentation

Fuzzy-Based Concept Learning Method: Exploiting Data With Fuzzy Conceptual Clustering

A semi-supervised clustering approach using labeled data

Semisupervised deep learning using consistency regularization and pseudolabels for hyperspectral image classification

CAFS: Class Adaptive Framework for Semi-Supervised Semantic Segmentation

Semi-Supervised Medical Image Classification Combined with Unsupervised Deep Clustering

HCL: Hierarchical Consistency Learning for Webly Supervised Fine-Grained Recognition

A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future Directions

Imbalance-Aware Discriminative Clustering for Unsupervised Semantic Segmentation