L. Demidova,E. Nikulchev,Yu. Sokolova
Abstract:The problem of development of the SVM classifier based on the modified particle swarm optimization has been considered. This algorithm carries out the simultaneous search of the kernel function type, values of the kernel function parameters and value of the regularization parameter for the SVM classifier. Such SVM classifier provides the high quality of data classification. The idea of particles' «regeneration» is put on the basis of the modified particle swarm optimization algorithm. At the realization of this idea, some particles change their kernel function type to the one which corresponds to the particle with the best value of the classification accuracy. The offered particle swarm optimization algorithm allows reducing the time expenditures for development of the SVM classifier. The results of experimental studies confirm the efficiency of this algorithm.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to develop an SVM classifier based on Modified Particle Swarm Optimization (M - PSO). Specifically, the author aims to simultaneously search for the kernel function type, kernel function parameter values, and regularization parameter values of the SVM classifier through the M - PSO algorithm, in order to improve the quality of data classification and reduce the time required to build the SVM classifier.
### Core elements of the problem
1. **Parameter selection of SVM classifier**:
- **Kernel function types**: including linear kernel, polynomial kernel, Radial Basis Function (RBF) kernel, and Sigmoid kernel.
- **Kernel function parameters**: for example, for the RBF kernel, \(\sigma\) needs to be determined; for the polynomial kernel, \(d\) needs to be determined, etc.
- **Regularization parameter \(C\)**: controls the balance between model complexity and classification error.
2. **Limitations of traditional methods**:
- The traditional Particle Swarm Optimization (PSO) algorithm usually fixes the kernel function type and optimizes the kernel function parameters and regularization parameters separately.
- This method is less efficient and requires multiple runs of the PSO algorithm to compare the performance of different kernel function types.
3. **Improved Particle Swarm Optimization algorithm**:
- An improved PSO algorithm is proposed, which can simultaneously search for the best kernel function type, kernel function parameters, and regularization parameters in one optimization process.
- A "particle regeneration" mechanism is introduced, that is, some particles will change their kernel function types according to the current optimal classification accuracy, thus accelerating the convergence to the global optimal solution.
### Formula representation
- **PSO velocity update formula**:
\[
v_{ij}^{t + 1}=\chi\left(v_{ij}^t+\phi_p r_p\left(p_{ij}-x_{ij}^t\right)+\phi_g r_g\left(g_j - x_{ij}^t\right)\right)
\]
where:
- \(v_{ij}^t\) is the velocity of the \(i\)-th particle in the \(j\)-th dimension;
- \(x_{ij}^t\) is the position of the \(i\)-th particle in the \(j\)-th dimension;
- \(p_{ij}\) is the historical optimal position of the \(i\)-th particle in the \(j\)-th dimension;
- \(g_j\) is the global optimal position;
- \(r_p\) and \(r_g\) are random numbers between [0, 1];
- \(\phi_p\) and \(\phi_g\) are personal and global acceleration coefficients;
- \(\chi\) is a compression factor.
- **Position update formula**:
\[
x_{ij}^{t + 1}=x_{ij}^t+v_{ij}^{t + 1}
\]
### Experimental results
Experimental studies show that the improved PSO algorithm exhibits higher classification accuracy and shorter computation time on multiple real - world datasets (such as breast cancer diagnosis, heart disease diagnosis, and credit scoring), which verifies the effectiveness of this method.
### Conclusion
By introducing the improved Particle Swarm Optimization algorithm, the paper successfully solves the multi - objective optimization problem in SVM classifier parameter selection, significantly improving the classification performance and reducing the computational cost.