E. G. Abramov,A. B. Komissarov,D. A. Kornyakov
Abstract:In this paper there is proposed a generalized version of the SVM for binary classification problems in the case of using an arbitrary transformation x -> y. An approach similar to the classic SVM method is used. The problem is widely explained. Various formulations of primal and dual problems are proposed. For one of the most important cases the formulae are derived in detail. A simple computational example is demonstrated. The algorithm and its implementation is presented in Octave language.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to improve the performance and flexibility of Support Vector Machines (SVMs) in binary classification problems. Specifically, the author proposes a generalized SVM method, called the Supporting Hyperplane Machine (SHM), to handle cases using arbitrary transformations. The following is a summary of the main content of the paper:
### 1. Research Background
The traditional Support Vector Machine (SVM) was proposed by Cortes and Vapnik and has become one of the powerful tools in the field of artificial neural networks due to its simplicity and high efficiency. However, the classical SVM method may not be flexible enough in some complex cases, especially when using arbitrary transformations.
### 2. Proposed Method
The paper proposes a generalized version of SVM, namely the Supporting Hyperplane Machine (SHM), for solving binary classification problems. The core idea of this method is to find a family of optimal hyperplanes given a training set, and these hyperplanes depend on the input vectors and the set of functions used.
### 3. Mathematical Model
The paper describes in detail how to construct SHM through the following steps:
- **Linear SHM**: For the linearly separable case, the objective function and constraint conditions are defined, and the solution is obtained by the Lagrange multiplier method.
- **Non - linear SHM**: The kernel trick is introduced to handle non - linear cases.
- **Soft - margin case**: A certain error is allowed to exist in order to minimize the structural risk.
### 4. Key Formulas
- **Objective function**: \[ \min_{W, b} \frac{1}{2} \|W\|^2 \]
- **Constraint conditions**: \[ y_i (W^T \phi(x_i) + b) \geq 1 \quad \forall i \]
- **Lagrange function**: \[ \mathcal{L}(W, b, \alpha) = \frac{1}{2} \|W\|^2 - \sum_{i = 1}^n \alpha_i [y_i (W^T \phi(x_i) + b) - 1] \]
### 5. Experimental Verification
The paper demonstrates the effectiveness of the SHM method through a simple computational example. In the experiment, a training set of 16 points is used, and two tabular functions are applied for transformation. The results show that the SHM method can effectively identify and support hyperplanes.
### 6. Application Prospects
The SHM method is not only applicable to linear classification problems, but can also be extended to non - linear classification problems through the kernel trick. In addition, the paper also explores the optimization problem in the soft - margin case, making this method more flexible and robust in practical applications.
In conclusion, this paper aims to further improve the performance and adaptability of SVM in binary classification problems by proposing the SHM method, especially when dealing with complex data transformations.