Abstract:Support vector regression (SVR) has garnered significant popularity over the past two decades owing to its wide range of applications across various fields. Despite its versatility, SVR encounters challenges when confronted with outliers and noise, primarily due to the use of the $\varepsilon$-insensitive loss function. To address this limitation, SVR with bounded loss functions has emerged as an appealing alternative, offering enhanced generalization performance and robustness. Notably, recent developments focus on designing bounded loss functions with smooth characteristics, facilitating the adoption of gradient-based optimization algorithms. However, it's crucial to highlight that these bounded and smooth loss functions do not possess an insensitive zone. In this paper, we address the aforementioned constraints by introducing a novel symmetric loss function named the HawkEye loss function. It is worth noting that the HawkEye loss function stands out as the first loss function in SVR literature to be bounded, smooth, and simultaneously possess an insensitive zone. Leveraging this breakthrough, we integrate the HawkEye loss function into the least squares framework of SVR and yield a new fast and robust model termed HE-LSSVR. The optimization problem inherent to HE-LSSVR is addressed by harnessing the adaptive moment estimation (Adam) algorithm, known for its adaptive learning rate and efficacy in handling large-scale problems. To our knowledge, this is the first time Adam has been employed to solve an SVR problem. To empirically validate the proposed HE-LSSVR model, we evaluate it on UCI, synthetic, and time series datasets. The experimental outcomes unequivocally reveal the superiority of the HE-LSSVR model both in terms of its remarkable generalization performance and its efficiency in training time.

A Novel Bounded Loss Framework for Support Vector Machines

A Novel Robust Smooth Support Vector Machine

Bounded quantile loss for robust support vector machines-based classification and regression

Flatheaded support vector machine for regression

A Novel Loss Function-based Support Vector Machine for Binary Classification

Robust Iterative Algorithm of Weighted Least Squares Support Vector Machine and Its Application in Spectral Analysis

M-estimator Based Support Vector Machine and Its Application

Asymmetric and robust loss function driven least squares support vector machine

Support vector machines with the ε -insensitive pinball loss function for uncertain data classification

Robust Support Vector Machine Using Least Median Loss Penalty

Generalized robust loss functions for machine learning

Support Vector Machine Classifier with Pinball Loss

CTSVM: A robust twin support vector machine with correntropy-induced loss function for binary classification problems

HawkEye: Advancing Robust Regression with Bounded, Smooth, and Insensitive Loss Function

Multiscale Support Vector Regression Method in Sobolev Spaces on Bounded Domains

Smooth pinball loss nonparallel support vector machine for robust classification

Sequential minimal optimization for SVM with pinball loss.

Improvement over Pinball Loss Support Vector Machine

RoBoSS: A Robust, Bounded, Sparse, and Smooth Loss Function for Supervised Learning

Improving Sparsity and Scalability in Regularized Nonconvex Truncated-Loss Learning Problems

Ramp Loss Linear Programming Support Vector Machine.