Abstract:Pseudo-labeling-based approaches are gaining prominence in Semi-Supervised Learning (SSL). Recent studies have identified that the key bottleneck in this methodology is addressing insufficient, incorrect, and imbalanced pseudo-labels. In this paper, we argue that the intrinsic problem behind this bottleneck is classifier bias, i.e., the classifier's prototypes suffer from poor uniformity. Further, inspired by neural collapse that reveals an optimal structure under supervised training scenarios, we address this classifier bias by utilizing an offline simplex Equiangular Tight Frame (ETF) classifier with maximally and equally separated prototypes. During the training phase, we maintain the prototypes of the classifier as fixed and concentrate on refining the feature encoder. Specifically, we integrate a straightforward clustering-based pseudo-labeling strategy with information maximization for feature learning. In practice, the fixed ETF classifier prevents the model from falling into a detrimental cycle, where a biased classifier exacerbates misaligned features, further perpetuating this bias. Furthermore, the clustering-based pseudo-labeling strategy reduces the dependency on complex threshold-adjusting mechanisms and effectively navigates the quantity-quality trade-off that plagues existing SSL methods. Leveraging these methodologies, we develop a simple yet powerful approach, termed ETF-SSL. Extensive experiments across Image, Text, and Audio datasets demonstrate that ETF-SSL can achieve competitive or superior performance compared to existing approaches. This success highlights the benefits of using a fixed ETF classifier in SSL and points to promising directions for future research in this area. The code is available at: https://github.com/yichenwang231/ETFSSL .

Robust Pseudo-Label Selection for Holistic Semi-Supervised Learning

Robust Semi-Supervised Learning when Not All Classes have Labels

Leveraging Local Variance for Pseudo-Label Selection in Semi-supervised Learning

Smooth Pseudo-Labeling

In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning

LaSSL: Label-Guided Self-Training for Semi-supervised Learning

Robust Deep Semi-Supervised Learning: A Brief Introduction

DP-SSL: Towards Robust Semi-supervised Learning with A Few Labeled Samples

On Pseudo-Labeling for Class-Mismatch Semi-Supervised Learning

SemiReward: A General Reward Model for Semi-supervised Learning

Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data

A Channel-ensemble Approach: Unbiased and Low-variance Pseudo-labels is Critical for Semi-supervised Classification

Towards Self-Adaptive Pseudo-Label Filtering for Semi-Supervised Learning

Learning Label Refinement and Threshold Adjustment for Imbalanced Semi-Supervised Learning

Meta-Semi: A Meta-learning Approach for Semi-supervised Learning.

Generalized Semi-Supervised Learning via Self-Supervised Feature Adaptation

Roll With the Punches: Expansion and Shrinkage of Soft Label Selection for Semi-supervised Fine-Grained Learning

Robust Graph-Based Semisupervised Learning for Noisy Labeled Data Via Maximum Correntropy Criterion

Neural collapse inspired semi-supervised learning with fixed classifier

Rethinking Pseudo-labeled Sample Mining for Semi-Supervised Object Detection

Class-Aware Contrastive Semi-Supervised Learning