Abstract:Semi-Supervised Support Vector Machine (S3VM) is one of the most popular methods for semi-supervised learning, which can make full use of plentiful, easily accessible unlabeled data. Balancing constraint is normally enforced in S3VM (denoted as BCS3VM) to avoid the harmful solution which assigns all or most of the unlabeled examples to one same label. Traditionally, non-linear BCS3VM is solved by sequential minimal optimization algorithm. Recently, a novel incremental learning algorithm (IL-BCS3VM) was proposed to scale up BCS3VM further. However, IL-BCS3VM needs to calculate the inverse of the linear system related to the support matrix, making the algorithm not scalable enough. To make BCS3VM be more practical in large-scale problems, in this paper, we propose a new scalable BCS3VM with accelerated triply stochastic gradients (denoted as TSG-BCS3VM). Specifically, to make the balancing constraint handle different proportions of positive and negative samples among labeled and unlabeled data, we propose a soft balancing constraint for S3VM. To make the algorithm scalable, we generate triply stochastic gradients by sampling labeled and unlabeled samples as well as the random features to update the solutions, where Quasi-Monte Carlo (QMC) sampling is utilized on random features to accelerate TSG-BCS3VM further. Our theoretical analysis shows that the convergence rate is O(1/√T) for both diminishing and constant learning rates where T is the number of iterations, which is much better than previous results thanks to the QMC method. Empirical results on a variety of benchmark datasets show that our algorithm not only has a good generalization performance but also enjoys better scalability than existing BCS3VM algorithms.

A class of semi-supervised support vector machines by DC programming

A Class of Smooth Semi-Supervised SVM by Difference of Convex Functions Programming and Algorithm

Distributed Online Semi-Supervised Support Vector Machine

An On-Line Learning Approach with Support Vector Dormain Classifier

Optimization meets Machine Learning: An Exact Algorithm for Semi-Supervised Support Vector Machines

A Multi-Class Large Margin Classifier

DCA for Sparse Quadratic Kernel-Free Least Squares Semi-Supervised Support Vector Machine

A New Semi-Supervised Classifier Based on Maximum Vector-Angular Margin

An Overview on Semi-Supervised Support Vector Machine

Cuts3vm: a fast semi-supervised svm algorithm.

New Incremental Learning Algorithm for Semi-Supervised Support Vector Machine

One new family of smooth semi-supervised support vector classifier based on Fourier series approximation technique

A New Support Vector Machine for Multi-class Classification

New unsupervised support vector machines

Robust Unsupervised and Semi-Supervised Bounded V - Support Vector Machines

Towards Practical Large Scale Non-Linear Semi-Supervised Learning with Balancing Constraints.

Unsupervised and Semi-Supervised Bounded V-Support Vector Machines with Polyhedral Perturbations

Unsupervised and Semi-Supervised Two-class Support Vector Machines

Unsupervised and Semi-supervised Support Vector Machines

Unsupervised and Semi-Supervised Bounded C-Support Vector Machines with Polyhedral Perturbations

Improving Semi-Supervised Support Vector Machines Through Unlabeled Instances Selection.