Kernel-based Regression Via a Novel Robust Loss Function and Iteratively Reweighted Least Squares
Dong Hongwei,Yang Liming
DOI: https://doi.org/10.1007/s10115-021-01554-8
IF: 2.7
2021-01-01
Knowledge and Information Systems
Abstract:Least squares kernel-based methods have been widely used in regression problems due to the simple implementation and good generalization performance. Among them, least squares support vector regression (LS-SVR) and extreme learning machine (ELM) are popular techniques. However, the noise sensitivity is a major bottleneck. To address this issue, a generalized loss function, called ℓ s -loss, is proposed in this paper. With the support of novel loss function, two kernel-based regressors are constructed by replacing the ℓ 2 -loss in LS-SVR and ELM with the proposed ℓ s -loss for better noise robustness. Important properties of ℓ s -loss, including robustness, asymmetry and asymptotic approximation behaviors, are verified theoretically. Moreover, iteratively reweighted least squares are utilized to optimize and interpret the proposed methods from a weighted viewpoint. The convergence of the proposal is proved, and detailed analyses of robustness are given. Experiments on both artificial and benchmark datasets confirm the validity of the proposed methods.