AllMatch: Exploiting All Unlabeled Data for Semi-Supervised Learning

Zhiyu Wu,Jinshi Cui

2024-07-09

Abstract:Existing semi-supervised learning algorithms adopt pseudo-labeling and consistency regulation techniques to introduce supervision signals for unlabeled samples. To overcome the inherent limitation of threshold-based pseudo-labeling, prior studies have attempted to align the confidence threshold with the evolving learning status of the model, which is estimated through the predictions made on the unlabeled data. In this paper, we further reveal that classifier weights can reflect the differentiated learning status across categories and consequently propose a class-specific adaptive threshold mechanism. Additionally, considering that even the optimal threshold scheme cannot resolve the problem of discarding unlabeled samples, a binary classification consistency regulation approach is designed to distinguish candidate classes from negative options for all unlabeled samples. By combining the above strategies, we present a novel SSL algorithm named AllMatch, which achieves improved pseudo-label accuracy and a 100% utilization ratio for the unlabeled data. We extensively evaluate our approach on multiple benchmarks, encompassing both balanced and imbalanced settings. The results demonstrate that AllMatch consistently outperforms existing state-of-the-art methods.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve

This paper aims to address two key issues in Semi-Supervised Learning (SSL):

1. **How to Improve the Quality of Pseudo-Labels**:
- Existing semi-supervised learning algorithms introduce supervision signals for unlabeled samples through pseudo-labeling and consistency regularization techniques. To overcome the inherent limitations of threshold-based pseudo-label methods, previous research has attempted to dynamically adjust the confidence threshold according to the model's learning state.
- This paper further reveals that classifier weights can reflect the differentiated learning states of different categories, and thus proposes a Class-Specific Adaptive Threshold (CAT) mechanism to improve the representation of the model's learning state.

2. **How to Utilize Low-Confidence Pseudo-Labels**:
- Even the optimal threshold strategy cannot solve the problem of discarding unlabeled samples. To address this, the paper designs a Binary Classification Consistency (BCC) method, which provides effective supervision signals for all unlabeled samples by distinguishing between candidate categories and negative categories.
- The BCC method can fully utilize low-confidence pseudo-labels, thereby achieving complete utilization of unlabeled data.

By combining the above strategies, the authors propose a new semi-supervised learning algorithm called AllMatch, which demonstrates superior performance compared to existing state-of-the-art methods across multiple benchmarks.