Abstract:Summary SSL (semi‐supervised learning) is widely used in machine learning, which leverages labeled and unlabeled data to improve model performance. SSL aims to optimize class mutual information, but noisy pseudo‐labels introduce false class information due to the scarcity of labels. Therefore, these algorithms often need significant training time to refine pseudo‐labels for performance improvement iteratively. To tackle this challenge, we propose a novel plug‐and‐play method named Accelerating semi‐supervised learning via contrastive learning (ASCL). This method combines contrastive learning with uncertainty‐based selection for performance improvement and accelerates the convergence of SSL algorithms. Contrastive learning initially emphasizes the mutual information between samples as a means to decrease dependence on pseudo‐labels. Subsequently, it gradually turns to maximizing the mutual information between classes, aligning with the objective of semi‐supervised learning. Uncertainty‐based selection provides a robust mechanism for acquiring pseudo‐labels. The combination of the contrastive learning module and the uncertainty‐based selection module forms a virtuous cycle to improve the performance of the proposed model. Extensive experiments demonstrate that ASCL outperforms state‐of‐the‐art methods in terms of both convergence efficiency and performance. In the experimental scenario where only one label is assigned per class in the CIFAR‐10 dataset, the application of ASCL to Pseudo‐label, UDA (unsupervised data augmentation for consistency training), and Fixmatch benefits substantial improvements in classification accuracy. Specifically, the results demonstrate notable improvements in respect of 16.32%, 6.9%, and 24.43% when compared to the original outcomes. Moreover, the required training time is reduced by almost 50%.

Towards the Out-of-Distribution Generalization of Contrastive Self-Supervised Learning

Boosting Unsupervised Contrastive Learning Using Diffusion-Based Data Augmentation from Scratch

ArCL: Enhancing Contrastive Learning with Augmentation-Robust Representations

Towards the Generalization of Contrastive Self-Supervised Learning

Data-Efficient Contrastive Self-supervised Learning: Most Beneficial Examples for Supervised Learning Contribute the Least

Is Self-Supervised Learning More Robust Than Supervised Learning?

On the effectiveness of out-of-distribution data in self-supervised long-tail learning.

A Survey on Contrastive Self-Supervised Learning

Self-supervised Learning: Generative or Contrastive

Weak Augmentation Guided Relational Self-Supervised Learning

Safe Semi-Supervised Contrastive Learning Using In-Distribution Data as Positive Examples

Contrastive Learning With Stronger Augmentations

Contrastive Learning with Synthetic Positives

Self-supervised Learning is More Robust to Dataset Imbalance

ASCL: Accelerating semi‐supervised learning via contrastive learning

Understanding the Role of Self-Supervised Learning in Out-of-Distribution Detection Task

Your Contrastive Learning Is Secretly Doing Stochastic Neighbor Embedding

On the Discriminability of Self-Supervised Representation Learning

Feature Augmentation for Self-supervised Contrastive Learning: A Closer Look

Understanding self-supervised Learning Dynamics without Contrastive Pairs

Augmented Contrastive Self-Supervised Learning for Audio Invariant Representations