Abstract:Semi-Supervised Learning (SSL) has been proved to be an effective way to leverage both labeled and unlabeled data at the same time. Recent semi-supervised approaches focus on deep neural networks and have achieved promising results on several benchmarks: CIFAR10, CIFAR100 and SVHN. However, most of their experiments are based on models trained from scratch instead of pre-trained models. On the other hand, transfer learning has demonstrated its value when the target domain has limited labeled data. Here comes the intuitive question: is it possible to incorporate SSL when fine-tuning a pre-trained model? We comprehensively study how SSL methods starting from pretrained models perform under varying conditions, including training strategies, architecture choice and datasets. From this study, we obtain several interesting and useful observations. While practitioners have had an intuitive understanding of these observations, we do a comprehensive emperical analysis and demonstrate that: (1) the gains from SSL techniques over a fully-supervised baseline are smaller when trained from a pre-trained model than when trained from random initialization, (2) when the domain of the source data used to train the pre-trained model differs significantly from the domain of the target task, the gains from SSL are significantly higher and (3) some SSL methods are able to advance fully-supervised baselines (like Pseudo-Label). We hope our studies can deepen the understanding of SSL research and facilitate the process of developing more effective SSL methods to utilize pre-trained models. Code is now available at github.

Semi-Supervised Learning for Signal Recognition with Sparsity and Robust Promotion

Semi-Supervised Learning via Swapped Prediction for Communication Signal Recognition

SemiNLL: A Framework of Noisy-Label Learning by Semi-Supervised Learning

Robust Deep Semi-Supervised Learning: A Brief Introduction

Semi-supervised learning with extremely sparse labeled data on multiple semi-supervised assumptions

Semi-supervised learning advances species recognition for aquatic biodiversity monitoring

SSwsrNet: A Semi-Supervised Few-Shot Learning Framework for Wireless Signal Recognition

DP-SSL: Towards Robust Semi-supervised Learning with A Few Labeled Samples

DeLaLA: Semisupervised Learning via Determinately Labeling and Kernelized Large Margin Projection

When Semi-Supervised Learning Meets Transfer Learning: Training Strategies, Models and Datasets.

Self-Supervised RF Signal Representation Learning for NextG Signal Classification with Deep Learning

Meta-Semi: A Meta-learning Approach for Semi-supervised Learning.

Reinforcement Learning-Guided Semi-Supervised Learning

SemiReward: A General Reward Model for Semi-supervised Learning

LaSSL: Label-Guided Self-Training for Semi-supervised Learning

FlexSSL : A Generic and Efficient Framework for Semi-Supervised Learning

Teacher/Student Deep Semi-Supervised Learning for Training with Noisy Labels

Scalable Graph-Based Semi-Supervised Learning Through Sparse Bayesian Model.

Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data.

CA-SSL: Class-Agnostic Semi-Supervised Learning for Detection and Segmentation

Boosting Semi-Supervised Face Recognition With Noise Robustness