Abstract:We show how, using linear-algebraic tools developed to prove Tverberg's theorem in combinatorial geometry, we can design new models of multi-class support vector machines (SVMs). These supervised learning protocols require fewer conditions to classify sets of points, and can be computed using existing binary SVM algorithms in higher-dimensional spaces, including soft-margin SVM algorithms. We describe how the theoretical guarantees of standard support vector machines transfer to these new classes of multi-class support vector machines. We give a new simple proof of a geometric characterization of support vectors for largest margin SVMs by Veelaert.

What problem does this paper attempt to address?

The paper primarily addresses the design and optimization of multi-class Support Vector Machines (SVMs), specifically how to use linear algebra tools to design new multi-class SVM models that require fewer conditions when classifying point sets. Specifically, the paper introduces a new multi-class SVM model by incorporating the relevant theories and techniques of the Tverberg theorem. This model can use existing binary SVM algorithms in high-dimensional spaces and handle multiple categories. The core contributions of the paper include: 1. **Proposal of New Models**: The authors propose two new types of multi-class SVM models—Simple Tverberg Multi-class Support Vector Machine (Simple TSVM) and Tverberg Multi-class Support Vector Machine (TSVM). Both models utilize linear algebra tools from the Tverberg theorem to reduce the prerequisites for classification and allow the use of existing binary SVM algorithms in higher dimensions. 2. **Transfer of Theoretical Guarantees**: The paper describes how the theoretical guarantees of standard support vector machines transfer to these new types of multi-class support vector machines. 3. **Geometric Property Proof**: A simple geometric property proof of the support vectors of the maximum margin SVM is provided. 4. **Model Complexity Analysis**: The paper also compares the computational complexity of different models and provides specific implementation details of the algorithms. 5. **Property Analysis**: For the proposed models, the paper analyzes their geometric properties, such as the use of half-spaces, the division of data space, and possible classification errors. 6. **Behavior Under Transformation**: Finally, the paper explores the behavior of the models under orthogonal transformations, i.e., the stability of the models when orthogonal transformations are applied to the input data. In summary, this paper aims to improve the design of multi-class SVM models by utilizing the Tverberg theorem and its related techniques, making them more efficient and flexible.

Tverberg's theorem and multi-class support vector machines

A Multi-Class Large Margin Classifier

On the proliferation of support vectors in high dimensions

A New Multi-Class Support Vector Machines

Stochastic Tverberg theorems and their applications in multi-class logistic regression, data separability, and centerpoints of data

Robust Twin Parametric Margin Support Vector Machine for Multiclass Classification

Support vector machines and Radon's theorem

A Tutorial on Support Vector Machines for Pattern Recognition

On support vector machines under a multiple-cost scenario

Multi-sphere Classification Algorithm Based on 1-SVM

Optimization meets Machine Learning: An Exact Algorithm for Semi-Supervised Support Vector Machines

Multiclass Graph-Based Large Margin Classifiers: Unified Approach for Support Vectors and Neural Networks

Multiview learning with twin parametric margin SVM

On $\ell_p$-Support Vector Machines and Multidimensional Kernels

Convex large margin training techniques: unsupervised, semi-supervised, and robust support vector machines

Support vector machines within a bivariate mixed-integer linear programming framework

Support Vector Machines with the Hard-Margin Loss: Optimal Training via Combinatorial Benders' Cuts

Enhancing Pattern Classification in Support Vector Machines through Matrix Formulation

Research and Analysis of Methods for Multiclass Support Vector Machines

An Improved Multiclass Algorithm For Svms

Robustness Verification of Support Vector Machines