A Robust Optimization Model for Nonlinear Support Vector Machine

Francesca Maggioni,Andrea Spinelli

2024-05-22

Abstract:In this paper, we present new optimization models for Support Vector Machine (SVM), with the aim of separating data points in two or more classes. The classification task is performed by means of nonlinear classifiers induced by kernel functions. Along with a deterministic formulation in which data are assumed to be perfectly known, we propose a robust extension with bounded-by-norm uncertainty sets. Robust optimization techniques allow to protect the models against uncertainties arising during the collection of real-world observations. Closed-form expressions for the bounds of the uncertainty sets in the feature space for typically used kernel functions have been derived. Extensive numerical results on real-world datasets show the benefits of the proposed classifiers in comparison with various alternatives in the SVM literature.

Optimization and Control

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper aims to propose a new optimization model to improve the performance of Support Vector Machines (SVM) in nonlinear classification tasks and enhance the robustness of the model to data uncertainty through robust optimization techniques. Specifically, the objectives of the paper include: 1. **Extending the Binary Linear SVM Method**: Extending the binary linear SVM method proposed by Liu & Potra (2009) to multi-class and nonlinear classification tasks. 2. **Introducing a Robust SVM Model**: Proposing a robust SVM model that considers the uncertainty in input data and uses norm-constrained uncertainty sets. 3. **Calculating the Uncertainty Set Boundaries in Feature Space**: Deriving closed-form expressions for the boundaries of uncertainty sets in feature space for commonly used kernel functions (such as the Gaussian kernel). 4. **Providing Extensive Numerical Experiments**: Conducting extensive numerical experiments based on real datasets to evaluate the proposed model and compare it with methods in existing literature. The main contribution of the paper lies in proposing a new robust optimization model that not only extends existing SVM methods but also provides closed-form expressions for the boundaries of uncertainty sets in feature space. These improvements enhance the robustness and accuracy of the classifier when facing data uncertainty.

A Robust Optimization Model for Nonlinear Support Vector Machine

A Novel Robust Smooth Support Vector Machine

Weighted Support Vector Machines for Regression and Its Application

A Novel Svm Modeling Approach For Highly Imbalanced And Overlapping Classification

Using Svm To Model And Control Nonlinear Dynamical Systems

A survey of robust optimization based machine learning with special reference to support vector machines

M-estimator Based Support Vector Machine and Its Application

Robust support vector machines via conic optimization

Mini-batch Quasi-Newton Optimization for Large Scale Linear Support Vector Regression

Data-driven Robust Optimization Based on Kernel Learning.

Homotopy continuation approaches for robust SV classification and regression

Theoretical aspects of robust SVM optimization in Banach spaces and Nash equilibrium interpretation

Non-smooth optimization algorithm to solve the LINEX soft support vector machine

Robustness Verification of Support Vector Machines

Robust Twin Parametric Margin Support Vector Machine for Multiclass Classification

Robust optimization for adversarial learning with finite sample complexity guarantees

Wasserstein Distributionally Robust Multiclass Support Vector Machine

Support Vector Machine-Based Nonlinear System Modeling and Control

Support vector machine adaptive control of nonlinear systems

Evaluating robustness of support vector machines with the Lagrangian dual approach