Abstract:<p>Semi-supervised learning has largely alleviated the strong demand for large amount of annotations in deep learning. However, most of the methods have adopted a common assumption that there is always labeled data from the same class of unlabeled data, which is impractical and restricted for real-world applications. In this research work, our focus is on semi-supervised learning when the categories of unlabeled data and labeled data are disjoint from each other. The main challenge is how to effectively leverage knowledge in labeled data to unlabeled data when they are independent from each other, and not belonging to the same categories. Previous state-of-the-art methods have proposed to construct pairwise similarity pseudo labels as supervising signals. However, two issues commonly inherent in these methods: (1) All of previous methods are comprised of multiple training phases, which makes it difficult to train the model in an end-to-end fashion. (2) Strong dependence on the quality of pairwise similarity pseudo labels limits the performance as pseudo labels are vulnerable to noise and bias. Therefore, we propose to exploit the use of self-supervision as auxiliary task during model training such that labeled data and unlabeled data will share the same set of surrogate labels and overall supervising signals can have strong regularization. By doing so, all modules in the proposed algorithm can be trained simultaneously, which will boost the learning capability as end-to-end learning can be achieved. Moreover, we propose to utilize local structure information in feature space during pairwise pseudo label construction, as local properties are more robust to noise. Extensive experiments have been conducted on three frequently used visual datasets, i.e., CIFAR-10, CIFAR-100 and SVHN, in this paper. Experiment results have indicated the effectiveness of our proposed algorithm as we have achieved new state-of-the-art performance for novel visual categories learning for these three datasets.</p>

Exploring Self-Supervised Regularization for Supervised and Semi-Supervised Learning

Semi-Supervised Learning for Fine-Grained Classification with Self-Training.

VISER: Visual Self-Regularization

Deep virtual adversarial self-training with consistency regularization for semi-supervised medical image classification

Semi-supervised Active Learning Based on Semantic-aware Crop Consistency

Self-Supervised Representation Learning with Meta Comprehensive Regularization

Semisupervised Semantic Segmentation of Remote Sensing Images With Consistency Self-Training

Regularizing Self-training for Unsupervised Domain Adaptation via Structural Constraints

A Simple Baseline for Semi-supervised Semantic Segmentation with Strong Data Augmentation

Semi-supervised semantic segmentation needs strong, varied perturbations

Source-free Semantic Regularization Learning for Semi-supervised Domain Adaptation

End-to-end novel visual categories learning via auxiliary self-supervision

Better Self-training for Image Classification Through Self-supervision

Supervised Image Classification with Self-Paced Regularization.

Colour augmentation for improved semi-supervised semantic segmentation

Self-supervised Pretraining of Visual Features in the Wild

Transformation-Consistent Self-Ensembling Model for Semisupervised Medical Image Segmentation

Consistency Regularization with Generative Adversarial Networks for Semi-Supervised Image Classification

Semi-supervised learning made simple with self-supervised clustering

Semisupervised deep learning using consistency regularization and pseudolabels for hyperspectral image classification

Self-Supervised Speaker Verification with Simple Siamese Network and Self-Supervised Regularization