Abstract:Recently, deep learning (DL) presents a promising performance in the joint classification of multimodal remote sensing (RS) data. However, most of the approaches adopt a supervised learning (SL) manner, where the discrimination capability is limited by the paucity of labeled samples. Though some attempts have been made to develop semi-supervised methods, they prefer to select the highly confident predictions as pseudo-ground truth and discard those unreliable ones. Actually, unreliable samples can also provide useful information, e.g., indicating the categories to which samples may belong and definitely not belong. Focused on this, a novel uncertainty-aware contrastive learning (UACL) method is proposed. Here, label uncertainty analysis (UA) based on multilevel probability estimation is first conducted to separate reliable and unreliable samples, which are then processed with a designed hybrid ("hard" or "soft") contrastive learning (CL) strategy. For reliable samples, the "hard" CL pushes the network to learn features that will minimize the intra-class distance while maximizing the inter-class distance, according to the pseudo-labels. For unreliable samples, the "soft" CL aims to learn the similarity and difference among samples, where the predicted class probabilities are queried to estimate a soft mask for an adaptive feature similarity measurement. Moreover, a multimodal spectral-spatial joint feature representation pipeline of triple branches, i.e., one spectral branch for hyperspectral images (HSIs) and two spatial branches for multimodal data, is also introduced. By jointly learning from both labeled and unlabeled samples, more discriminative spectral-spatial feature representation will lead to a further boost in classification performance. Extensive experiments on four well-known multimodal datasets prove the effectiveness of the proposed semi-supervised classification method. Codes are available at https://github.com/Ding-Kexin/UACL.

Multi-expert contrastive learning for remote sensing long-tailed image classification

Dual-branch Branch Networks Based on Contrastive Learning for Long-Tailed Remote Sensing

EME: Energy-based Multi-Expert Model for Long-tailed Remote Sensing Image Classification

Bt-Vmf Contrastive and Collaborative Learning for Long-Tailed Visual Recognition

DECOR: Dynamic Decoupling and Multi-Objective Optimization for Long-tailed Remote Sensing Image Classification

Long-Tailed Object Detection for Multimodal Remote Sensing Images

Targeted Supervised Contrastive Learning for Long-Tailed Recognition

DECOR: Dynamic Decoupling and Multiobjective Optimization for Long-Tailed Remote Sensing Image Classification

Long-tailed Detection Based on Multi-Expert Aggregation

Contrastive Learning Based Hybrid Networks for Long-Tailed Image Classification

Multimodal Contrastive Learning for Remote Sensing Image Feature Extraction Based on Relaxed Positive Samples

Lightweight Channel Attention and Multiscale Feature Fusion Discrimination for Remote Sensing Scene Classification

Ternary Modality Contrastive Learning for Hyperspectral and LiDAR Data Classification

Contrastive Learning Based on Multiscale Hard Features for Remote-Sensing Image Scene Classification.

Balanced Contrastive Learning for Long-Tailed Visual Recognition

Enhanced multi-branch learning for long-tailed image recognition

Unsupervised Representation High-Resolution Remote Sensing Image Scene Classification Via Contrastive Learning Convolutional Neural Network

A Combination of Lie Group Machine Learning and Deep Learning for Remote Sensing Scene Classification Using Multi-Layer Heterogeneous Feature Extraction and Fusion

Uncertainty-Aware Contrastive Learning for Semi-Supervised Classification of Multimodal Remote Sensing Images

DEEP LEARNING TRAINING WITH UNBALANCE SAMPLE DISTRIBUTION FOR REMOTE SENSING IMAGE SEGMENTATION

A Coupled Multi-Task Feature Boosting Method for Remote Sensing Scene Classification