Abstract:Binary classification tasks with imbalanced classes pose significant challenges in machine learning. Traditional classifiers often struggle to accurately capture the characteristics of the minority class, resulting in biased models with subpar predictive performance. In this paper, we introduce a novel approach to tackle this issue by leveraging Universum points to support the minority class within quadratic twin support vector machine models. Unlike traditional classifiers, our models utilize quadratic surfaces instead of hyperplanes for binary classification, providing greater flexibility in modeling complex decision boundaries. By incorporating Universum points, our approach enhances classification accuracy and generalization performance on imbalanced datasets. We generated four artificial datasets to demonstrate the flexibility of the proposed methods. Additionally, we validated the effectiveness of our approach through empirical evaluations on benchmark datasets, showing superior performance compared to conventional classifiers and existing methods for imbalanced classification.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: in binary classification tasks, when the data classes are imbalanced, traditional classifiers have difficulty accurately capturing the characteristics of the minority class, resulting in model bias and a decline in prediction performance. Specifically, the paper aims to support the minority class by introducing Universum points and utilize the Quadratic Surface Twin Support Vector Machine (QTSVM) to improve the classification accuracy and generalization performance on imbalanced datasets. ### Problem Background In machine learning, the class imbalance problem is a significant challenge in binary classification tasks. When the number of samples in one class is much larger than that in the other class, traditional classifiers tend to be biased towards the majority class, leading to poor prediction performance for the minority class. This not only reduces the overall accuracy of the model but may also cause distortion of the decision boundary, especially in critical areas such as fraud detection or medical diagnosis, where the consequences of misclassification can be very serious. ### Method Proposed in the Paper To solve the above - mentioned problems, the paper proposes a new method, namely **Imbalanced Universum Quadratic Twin Support Vector Machine (Im - U - QTSVM)**. The main innovative points of this method include: 1. **Using quadratic surfaces instead of hyperplanes**: Different from the traditional Support Vector Machine (SVM), QTSVM uses quadratic surfaces for classification, providing greater flexibility to model complex decision boundaries. 2. **Introducing Universum points**: Universum points refer to samples that do not belong to any class, and they can provide additional information to help the model better understand the data distribution. By introducing these points, especially in the minority class, the generalization ability of the model can be enhanced. 3. **Handling class imbalance**: To deal with the class imbalance problem, the paper proposes randomly under - sampling the majority class and introducing an appropriate amount of Universum points in the minority class to ensure that the optimization problem remains unbiased. 4. **Adding a regularization term**: To improve the stability and generalization ability of the model, the paper also adds an ℓ2 regularization term to the Hessian matrix. ### Formula Representation The following is the optimization problem formula for the Im - U - QTSVM model: For the minority class (assumed to be the positive class): \[ \min \frac{1}{2} \sum_{i \in I_1} \left( \frac{1}{2} x_i^T W_1 x_i + b_1^T x_i + c_1 \right)^2 + \frac{1}{2} C_1 |I_1| \sum_{i = 1}^{|\mathcal{I}_1|} \xi_{1i} + \frac{1}{2} C_{\hat{u}g} \sum_{j = 1}^{g} \psi_{1j} + \frac{1}{2} \lambda_1 \sum_{i \leq j} W_{1ij}^2 \] Constraints: \[ -\left( \frac{1}{2} \tilde{x}_i^T W_1 \tilde{x}_i + b_1^T \tilde{x}_i + c_1 \right) \geq 1 - \xi_{1i}, \quad i = 1,\ldots, |I_1| \] \[ \left( \frac{1}{2} \hat{u}_j^T W_1 \hat{u}_j + b_1^T \hat{u}_j + c_1 \right) \geq - 1+\varepsilon - \psi_{1j}, \quad j = 1,\ldots, g \] For the majority class: \[ \min \frac{1}{2} \sum_{i \in I_2} \left( \frac{1}{2} \bar{x}_i^T W_2 \bar{x}_i + b_2^T \bar{x}_i + c_2 \right)^2 + \frac{1}{2} C_2

Kernel-Free Universum Quadratic Surface Twin Support Vector Machines for Imbalanced Data

A Novel Svm Modeling Approach For Highly Imbalanced And Overlapping Classification

Sparse Universum Quadratic Surface Support Vector Machine Models for Binary Classification

Homocentric quadratic surfaces and maximum margin approach for imbalanced data classification

An Unbalanced Dataset Classification Approach Based On V-Support Vector Machine

Intuitionistic Fuzzy Universum Twin Support Vector Machine for Imbalanced Data

Kreĭn twin support vector machines for imbalanced data classification

An Effective Multiclass Twin Hypersphere Support Vector Machine and Its Practical Engineering Applications

Hybrid SVM algorithm oriented to classifying imbalanced datasets

Multi-task twin support vector machine with Universum data

Nonlinear kernel-free quadratic hyper-surface support vector machine with 0-1 loss function

Weighted Least Squares Twin Support Vector Machine with Fuzzy Rough Set Theory for Imbalanced Data Classification

Intuitionistic Fuzzy Weighted Least Squares Twin SVMs

Kernel-Free Quadratic Surface Regression for Multi-Class Classification

Granular Ball Twin Support Vector Machine with Universum Data

A Classfication Method For Imbalance Data Set Based on Kernel SMOTE

Improved SVM algorithm for imbalanced dataset classification

Optimizing Kernel Transformations to Handle Binary Class Imbalanced Dataset Classification

Least squares KNN-based weighted multiclass twin SVM

Multiview learning with twin parametric margin SVM

A Novel Imbalanced Data Classification Method Based on Weakly Supervised Learning for Fault Diagnosis