Fast Support Vector Machine with Low-Computational Complexity for Large-Scale Classification
Huajun Wang,Zhibin Zhu,Yuanhai Shao
DOI: https://doi.org/10.1109/tsmc.2024.3375021
2024-01-01
IEEE Transactions on Systems Man and Cybernetics Systems
Abstract:Support vector machine (SVM) is a popular supervised machine learning classifier and has found extensive applied in many fields, including biological sciences, disease detection, health and clinical sciences, cancer classification, and more. However, the major challenge faced by SVM is its high-computational complexity, which becomes a bottleneck for large-scale SVM. To reduce computational complexity, we design a novel truncated squared loss function to get the novel SVM $(L_{{\rm tsl}}$ -SVM), and is a challenging model due to its nonconvex and nonsmooth characteristics. To solve $L_{{\rm tsl}}$ -SVM, we present new concept of proximal stationary point to establish its optimality theory. Using this theory, we then develop a novel and fast alternating direction method of multipliers in terms of low-computational complexity to address $L_{{\rm tsl}}$ -SVM and our new proposed algorithm achieve global convergence. Finally, numerical experiments have verified the superior performance of our developed method in terms of classification accuracy, number of support vectors and computational speed when compared to other eight leading solvers. For instance, when solving the real dataset with more than $10^{7}$ samples, our developed method only takes 18.89 s, significantly outperforming other solvers that require at least 589.8 s.