Nonlinear kernel-free quadratic hyper-surface support vector machine with 0-1 loss function

Mingyang Wu,Zhixia Yang,Junyou Ye

2024-04-16

Abstract:For the binary classification problem, a novel nonlinear kernel-free quadratic hyper-surface support vector machine with 0-1 loss function (QSSVM$_{0/1}$) is proposed. Specifically, the task of QSSVM$_{0/1}$ is to seek a quadratic separating hyper-surface to divide the samples into two categories. And it has better interpretability than the methods using kernel functions, since each feature of the sample acts both independently and synergistically. By introducing the 0-1 loss function to construct the optimization model makes the model obtain strong sample sparsity. The proximal stationary point of the optimization problem is defined by the proximal operator of the 0-1 loss function, which figures out the problem of non-convex discontinuity of the optimization problem due to the 0-1 loss function. A new iterative algorithm based on the alternating direction method of multipliers (ADMM) framework is designed to solve the optimization problem, which relates to the working set defined by support vectors. The computational complexity and convergence of the algorithm are discussed. Numerical experiments on 4 artificial datasets and 14 benchmark datasets demonstrate that our QSSVM$_{0/1}$ achieves higher classification accuracy, fewer support vectors and less CPU time cost than other state-of-the-art methods.

Optimization and Control

What problem does this paper attempt to address?

The paper aims to address the binary classification problem and proposes a novel nonlinear kernel-free quadratic hypersurface support vector machine (QSSVM 0/1), which combines the advantages of the 0-1 loss function. Specifically, the objective of QSSVM 0/1 is to find a quadratic separating hypersurface to partition the samples. Compared to methods using kernel functions, this approach offers better interpretability as each feature acts both independently and synergistically. By introducing the 0-1 loss function to construct the optimization model, the model achieves stronger sample sparsity. Additionally, the paper defines an approximate stationary point for the optimization problem and designs a new iterative algorithm based on the alternating direction method of multipliers (ADMM) framework to solve the optimization problem. Numerical experiments demonstrate that QSSVM 0/1 achieves higher classification accuracy, fewer support vectors, and shorter CPU time costs on 4 artificial datasets and 14 benchmark datasets, outperforming other state-of-the-art methods.