Abstract:Semi-supervised learning (SSL) has attracted enormous attention due to its vast potential of mitigating the dependence on large labeled datasets. The latest methods (e.g., FixMatch) use a combination of consistency regularization and pseudo-labeling to achieve remarkable successes. However, these methods all suffer from the waste of complicated examples since all pseudo-labels have to be selected by a high threshold to filter out noisy ones. Hence, the examples with ambiguous predictions will not contribute to the training phase. For better leveraging all unlabeled examples, we propose two novel techniques: Entropy Meaning Loss (EML) and Adaptive Negative Learning (ANL). EML incorporates the prediction distribution of non-target classes into the optimization objective to avoid competition with target class, and thus generating more high-confidence predictions for selecting pseudo-label. ANL introduces the additional negative pseudo-label for all unlabeled data to leverage low-confidence examples. It adaptively allocates this label by dynamically evaluating the top-k performance of the model. EML and ANL do not introduce any additional parameter and hyperparameter. We integrate these techniques with FixMatch, and develop a simple yet powerful framework called FullMatch. Extensive experiments on several common SSL benchmarks (CIFAR-10/100, SVHN, STL-10 and ImageNet) demonstrate that FullMatch exceeds FixMatch by a large margin. Integrated with FlexMatch (an advanced FixMatch-based framework), we achieve state-of-the-art performance. Source code is at <a class="link-external link-https" href="https://github.com/megvii-research/FullMatch" rel="external noopener nofollow">this https URL</a>.

FlexSSL : A Generic and Efficient Framework for Semi-Supervised Learning

LaSSL: Label-Guided Self-Training for Semi-supervised Learning

Meta-Semi: A Meta-learning Approach for Semi-supervised Learning.

Robust Semi-Supervised Learning when Not All Classes have Labels

SemiReward: A General Reward Model for Semi-supervised Learning

Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data

Improving Barely Supervised Learning by Discriminating Unlabeled Samples with Super-Class

Generalized Semi-Supervised Learning via Self-Supervised Feature Adaptation

Reinforcement Learning-Guided Semi-Supervised Learning

FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling

When Semi-Supervised Learning Meets Transfer Learning: Training Strategies, Models and Datasets.

Federated Semi-Supervised Learning with Annotation Heterogeneity

Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data.

A generic self-supervised learning (SSL) framework for representation learning from spectra-spatial feature of unlabeled remote sensing imagery

CA-SSL: Class-Agnostic Semi-Supervised Learning for Detection and Segmentation

FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning

Learning What and Where to Learn: A New Perspective on Self-supervised Learning

Label Diagnosis Through Self Tuning for Web Image Search

DeLaLA: Semisupervised Learning via Determinately Labeling and Kernelized Large Margin Projection

Robust Pseudo-Label Selection for Holistic Semi-Supervised Learning

AuxMix: Semi-Supervised Learning with Unconstrained Unlabeled Data