Abstract:Semi-Supervised Learning (SSL) seeks to leverage large amounts of non-annotated data along with the smallest amount possible of annotated data in order to achieve the same level of performance as if all data were annotated. A fruitful method in SSL is Pseudo-Labeling (PL), which, however, suffers from the important drawback that the associated loss function has discontinuities in its derivatives, which cause instabilities in performance when labels are very scarce. In the present work, we address this drawback with the introduction of a Smooth Pseudo-Labeling (SP L) loss function. It consists in adding a multiplicative factor in the loss function that smooths out the discontinuities in the derivative due to thresholding. In our experiments, we test our improvements on FixMatch and show that it significantly improves the performance in the regime of scarce labels, without addition of any modules, hyperparameters, or computational overhead. In the more stable regime of abundant labels, performance remains at the same level. Robustness with respect to variation of hyperparameters and training parameters is also significantly improved. Moreover, we introduce a new benchmark, where labeled images are selected randomly from the whole dataset, without imposing representation of each class proportional to its frequency in the dataset. We see that the smooth version of FixMatch does appear to perform better than the original, non-smooth implementation. However, more importantly, we notice that both implementations do not necessarily see their performance improve when labeled images are added, an important issue in the design of SSL algorithms that should be addressed so that Active Learning algorithms become more reliable and explainable.

SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learning

Roll With the Punches: Expansion and Shrinkage of Soft Label Selection for Semi-supervised Fine-Grained Learning

AllMatch: Exploiting All Unlabeled Data for Semi-Supervised Learning

FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning

Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data

OTAMatch: Optimal Transport Assignment with PseudoNCE for Semi-supervised Learning

HyperMatch: Noise-Tolerant Semi-Supervised Learning Via Relaxed Contrastive Constraint

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

DualMatch: Robust Semi-Supervised Learning with Dual-Level Interaction

Smooth Pseudo-Labeling

Enhancing Sample Utilization Through Sample Adaptive Augmentation in Semi-Supervised Learning

Class-Imbalanced Semi-Supervised Learning with Adaptive Thresholding.

A Channel-ensemble Approach: Unbiased and Low-variance Pseudo-labels is Critical for Semi-supervised Classification

Adaptive Weighted Losses With Distribution Approximation for Efficient Consistency-Based Semi-Supervised Learning

Leveraging Local Variance for Pseudo-Label Selection in Semi-supervised Learning

Rethinking Pseudo-labeled Sample Mining for Semi-Supervised Object Detection

Learning Label Refinement and Threshold Adjustment for Imbalanced Semi-Supervised Learning

Semi-Supervised Learning via Weight-aware Distillation under Class Distribution Mismatch

Class-Aware Contrastive Semi-Supervised Learning

MutexMatch: Semi-Supervised Learning With Mutex-Based Consistency Regularization

Learning sample-aware threshold for semi-supervised learning