Abstract:<p>Semi-supervised learning has largely alleviated the strong demand for large amount of annotations in deep learning. However, most of the methods have adopted a common assumption that there is always labeled data from the same class of unlabeled data, which is impractical and restricted for real-world applications. In this research work, our focus is on semi-supervised learning when the categories of unlabeled data and labeled data are disjoint from each other. The main challenge is how to effectively leverage knowledge in labeled data to unlabeled data when they are independent from each other, and not belonging to the same categories. Previous state-of-the-art methods have proposed to construct pairwise similarity pseudo labels as supervising signals. However, two issues commonly inherent in these methods: (1) All of previous methods are comprised of multiple training phases, which makes it difficult to train the model in an end-to-end fashion. (2) Strong dependence on the quality of pairwise similarity pseudo labels limits the performance as pseudo labels are vulnerable to noise and bias. Therefore, we propose to exploit the use of self-supervision as auxiliary task during model training such that labeled data and unlabeled data will share the same set of surrogate labels and overall supervising signals can have strong regularization. By doing so, all modules in the proposed algorithm can be trained simultaneously, which will boost the learning capability as end-to-end learning can be achieved. Moreover, we propose to utilize local structure information in feature space during pairwise pseudo label construction, as local properties are more robust to noise. Extensive experiments have been conducted on three frequently used visual datasets, i.e., CIFAR-10, CIFAR-100 and SVHN, in this paper. Experiment results have indicated the effectiveness of our proposed algorithm as we have achieved new state-of-the-art performance for novel visual categories learning for these three datasets.</p>

Self-Supervised Self-Supervision by Combining Deep Learning and Probabilistic Logic

Deep Probabilistic Logic: A Unifying Framework for Indirect Supervision.

Automated Graph Self-supervised Learning via Multi-teacher Knowledge Distillation

Self-distillation and self-supervision for partial label learning

On the Marginal Benefit of Active Learning: Does Self-Supervision Eat Its Cake?

Self-Adaptive Training: Bridging Supervised and Self-Supervised Learning.

Self-Supervision Can Be a Good Few-Shot Learner

Dlsa: Semi-supervised Partial Label Learning Via Dependence-Maximized Label Set Assignment

Self-supervised Learning for Semi-supervised Temporal Language Grounding

Self-supervised Knowledge Distillation for Complementary Label Learning

A Probabilistic Model Behind Self-Supervised Learning

Self-Tuning for Data-Efficient Deep Learning

Source-free Domain Adaptation Via Dynamic Pseudo Labeling and Self-supervision

End-to-end novel visual categories learning via auxiliary self-supervision

Deep Supervision with Intermediate Concepts

Self-Supervised Steering and Path Labeling for Autonomous Driving

Self Supervision to Distillation for Long-Tailed Visual Recognition

Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Experts

Boosting Supervision with Self-Supervision for Few-shot Learning

Co-learning: Learning from Noisy Labels with Self-supervision

Self-directed Machine Learning