Abstract:Recently, deep learning (DL) presents a promising performance in the joint classification of multimodal remote sensing (RS) data. However, most of the approaches adopt a supervised learning (SL) manner, where the discrimination capability is limited by the paucity of labeled samples. Though some attempts have been made to develop semi-supervised methods, they prefer to select the highly confident predictions as pseudo-ground truth and discard those unreliable ones. Actually, unreliable samples can also provide useful information, e.g., indicating the categories to which samples may belong and definitely not belong. Focused on this, a novel uncertainty-aware contrastive learning (UACL) method is proposed. Here, label uncertainty analysis (UA) based on multilevel probability estimation is first conducted to separate reliable and unreliable samples, which are then processed with a designed hybrid ("hard" or "soft") contrastive learning (CL) strategy. For reliable samples, the "hard" CL pushes the network to learn features that will minimize the intra-class distance while maximizing the inter-class distance, according to the pseudo-labels. For unreliable samples, the "soft" CL aims to learn the similarity and difference among samples, where the predicted class probabilities are queried to estimate a soft mask for an adaptive feature similarity measurement. Moreover, a multimodal spectral-spatial joint feature representation pipeline of triple branches, i.e., one spectral branch for hyperspectral images (HSIs) and two spatial branches for multimodal data, is also introduced. By jointly learning from both labeled and unlabeled samples, more discriminative spectral-spatial feature representation will lead to a further boost in classification performance. Extensive experiments on four well-known multimodal datasets prove the effectiveness of the proposed semi-supervised classification method. Codes are available at https://github.com/Ding-Kexin/UACL.

A novel semi-supervised hyperspectral image classification approach based on spatial neighborhood information and classifier combination

A Hyperspectral Image Classification Method Based on the Nonlocal Attention Mechanism of a Multiscale Convolutional Neural Network.

Semi-supervised Tri_training Hyperspectral Image Classification Approach Based on Spatial Neighborhood Information

Image classification of hyperspectral remote sensing using semi-supervised learning algorithm

A Spectral-Texture Kernel-Based Classification Method for Hyperspectral Images.

An efficient semi-supervised classification approach for hyperspectral imagery

A Novel Tri-Training Technique For Semi-Supervised Classification Of Hyperspectral Images Based On Diversity Measurement

Semi-supervised spectral-spatial weighted classification of hyperspectral image based on SVMSLR framework

Unified active and semi-supervised learning for hyperspectral image classification

Semisupervised deep learning using consistency regularization and pseudolabels for hyperspectral image classification

Unsupervised Hyperspectral Image Classification Method Based on Constrained Linear Discriminant Analysis

Semisupervised Spectral–Spatial Classification of Hyperspectral Imagery with Affinity Scoring

Semisupervised Hyperspectral Image Classification Network Based on Pseudo-Label and Spatial–Spectral Convolution

A 3-stage Spectral-spatial Method for Hyperspectral Image Classification

Semisupervised Discriminative Random Field for Hyperspectral Image Classification

Spectral–Spatial Prototype Learning-Based Nearest Neighbor Classifier for Hyperspectral Images

Uncertainty-Aware Contrastive Learning for Semi-Supervised Classification of Multimodal Remote Sensing Images

Semi-Supervised Hyperspectral Image Classification Based on Multiscale Spectral–Spatial Graph Attention Network

Nearest Neighbor-Based Contrastive Learning for Hyperspectral and LiDAR Data Classification

Self-Supervised Learning With Multiscale Densely Connected Network for Hyperspectral Image Classification

Dual-Stream Class-Adaptive Network for Semi-Supervised Hyperspectral Image Classification