Kernel-free Reduced Quadratic Surface Support Vector Machine with 0-1 Loss Function and L _p -norm Regularization

Mingyang Wu,Zhixia Yang
DOI: https://doi.org/10.1007/s40745-024-00573-w
2024-01-01
Annals of Data Science
Abstract:This paper presents a novel nonlinear binary classification method, namely the kernel-free reduced quadratic surface support vector machine with 0-1 loss function and L _p -norm regularization (L _p -RQSSVM _0/1 ). It uses kernel-free trick aimed at finding a reduced quadratic surface to separate samples, without considering the cross terms in quadratic form. This saves computational costs and provides better interpretability than methods using kernel functions. In addition, adding the 0-1 loss function and L _p -norm regularization to construct our L _p -RQSSVM _0/1 enables sample sparsity and feature sparsity. The support vector (SV) of L _p -RQSSVM _0/1 is defined, and it is derived that all SVs fall on the support hypersurfaces. Moreover, the optimality condition is explored theoretically, and a new iterative algorithm based on the alternating direction method of multipliers (ADMM) framework is used to solve our L _p -RQSSVM _0/1 on the selected working set. The computational complexity and convergence of the algorithm are discussed. Furthermore, numerical experiments demonstrate that our L _p -RQSSVM _0/1 achieves better classification accuracy, less SVs, and higher computational efficiency than other methods on most datasets. It also has feature sparsity under certain conditions.
What problem does this paper attempt to address?