Abstract:It is usually expected that learning performance can be improved by exploiting unlabeled data, particularly when the number of labeled data is limited. However, it has been reported that, in some cases existing semi-supervised learning approaches perform even worse than supervised ones which only use labeled data. For this reason, it is desirable to develop safe semi-supervised learning approaches that will not significantly reduce learning performance when unlabeled data are used. This paper focuses on improving the safeness of semi-supervised support vector machines (S3VMs). First, the S3VM-us approach is proposed. It employs a conservative strategy and uses only the unlabeled instances that are very likely to be helpful, while avoiding the use of highly risky ones. This approach improves safeness but its performance improvement using unlabeled data is often much smaller than S3VMs. In order to develop a safe and well-performing approach, we examine the fundamental assumption of S3VMs, i.e., low-density separation. Based on the observation that multiple good candidate low-density separators may be identified from training data, safe semi-supervised support vector machines (S4VMs) are here proposed. This approach uses multiple low-density separators to approximate the ground-truth decision boundary and maximizes the improvement in performance of inductive SVMs for any candidate separator. Under the assumption employed by S3VMs, it is here shown that S4VMs are provably safe and that the performance improvement using unlabeled data can be maximized. An out-of-sample extension of S4VMs is also presented. This extension allows S4VMs to make predictions on unseen instances. Our empirical study on a broad range of data shows that the overall performance of S4VMs is highly competitive with S3VMs, whereas in contrast to S3VMs which hurt performance significantly in many cases, S4VMs rarely perform worse than inductive SVMs.

Semi-Supervised AUC Optimization Without Guessing Labels of Unlabeled Data.

Weakly Supervised AUC Optimization: A Unified Partial AUC Approach

AUC Optimization from Multiple Unlabeled Datasets.

Generalized entropy based semi-supervised learning

Optimal and Safe Estimation for High-Dimensional Semi-Supervised Learning

AUL is a Better Optimization Metric in PU Learning

Cutting the Software Building Efforts in Continuous Integration by Semi-Supervised Online AUC Optimization.

Robust AUC optimization under the supervision of clean data

Safe semi-supervised learning: a brief introduction

Towards Making Unlabeled Data Never Hurt

Exploiting Unlabeled Data via Partial Label Assignment for Multi-Class Semi-Supervised Learning

Unsupervised Domain Adaptive Salient Object Detection Through Uncertainty-Aware Pseudo-Label Learning

Learning Semi-Supervised Representation Towards a Unified Optimization Framework for Semi-Supervised Learning

Improving Semi-Supervised Support Vector Machines Through Unlabeled Instances Selection.

Muffled Semi-Supervised Learning

Estimate Unlabeled-Data-Distribution for Semi-supervised PU Learning

Meta-Semi: A Meta-learning Approach for Semi-supervised Learning.

Self Adaptive Threshold Pseudo-labeling and Unreliable Sample Contrastive Loss for Semi-supervised Image Classification

Semi-Supervised U-statistics

Enhancing Sample Utilization Through Sample Adaptive Augmentation in Semi-Supervised Learning

Adaptive Semi-Supervised Mixup with Implicit Label Learning and Sample Ratio Balancing