Non-symmetric Support Vector Machines

Jianfeng Feng

DOI: https://doi.org/10.1007/3-540-45720-8_49

2001-01-01

Abstract:A novel approach to calculate the generalization error of the support vector machines and a new support vector machine-non-symmatic support vector machine-is proposedhere. Our results are based upon the extreme value theory and both the mean and variance of the generalization error are exactly ontained.

What problem does this paper attempt to address?

Flatheaded support vector machine for regression

ShaoChao Sun,Dao Huang

DOI: https://doi.org/10.1166/asl.2013.4907

2013-01-01

Advanced Science Letters

Abstract:In this paper, one robust non-convex loss function is constructed by combining two differentiable convex functions because non-convex loss functions own advantage over convex ones in robustness and generalization performance for support vector regression. With this non-convex loss function, a flatheaded support vector regression (FSVR) is proposed and then the concave-convex procedure is used to solve the FSVR by transforming the non-convex problem into a sequence of convex ones. The FSVR integrates the advantage of standard SVR and weighted SVR, in other words, it can not only obtain better sparseness property but also restrain the outliers of training samples. Experiments have been done on artificial and benchmark datasets and the results show the effectiveness of the proposed FSVR. © 2013 American Scientific Publishers. All rights reserved.
A Novel Robust Smooth Support Vector Machine

Shao Chao Sun,Dao Huang

DOI: https://doi.org/10.4028/www.scientific.net/amm.148-149.1438

2011-01-01

Applied Mechanics and Materials

Abstract:In this paper, we propose a new type of ε-insensitive loss function, called as ε-insensitive Fair estimator. With this loss function we can obtain better robustness and sparseness. To enhance the learning speed ,we apply the smoothing techniques that have been used for solving the support vector machine for classification, to replace the ε-insensitive Fair estimator by an accurate smooth approximation. This will allow us to solve ε-SFSVR as an unconstrained minimization problem directly. Based on the simulation results, the proposed approach has fast learning speed and better generalization performance whether outliers exist or not.
The generalization error of the symmetric and scaled support vector machines.

J Feng,P Williams

DOI: https://doi.org/10.1109/72.950155

2001-01-01

Abstract:It is generally believed that the support vector machine (SVM) optimizes the generalization error and outperforms other learning machines. We show analytically, by concrete examples in the one dimensional case, that the SVM does improve the mean and standard deviation of the generalization error by a constant factor, compared to the worst learning machine. Our approach is in terms of the extreme value theory and both the mean and variance of the generalization errors are calculated exactly for all the cases considered. We propose a new version of the SVM , called the scaled SVM, which can further reduce the mean of the generalization error of the SVM.
Weighted Support Vector Machines for Regression and Its Application

Shuxin Du

2004-01-01

Abstract:In the standard support vector machines for regression, the required error of regression estimation and the penalty for violation of the required error are equally considered for every training sample, which is unsuitable in case there exists significant difference among the training samples. In the proposed weighted support vector machines, each training sample had different approximation error requirement and different penalty. The dual quadratic optimization of weighted support vector machines for regression and its sequential minimal optimization (SMO) algorithms were given in detail. The experiments on the measurement of gasoline octane numbers by near-infrared spectroscopy, where the importance of each training sample was characterized by the geometrical distance from the test sample, show that the measurement accuracy is improved with the proposed weighted support vector machines.
M-estimator Based Support Vector Machine and Its Application

BAO Xin,DAI Liankui

DOI: https://doi.org/10.3321/j.issn:0438-1157.2009.07.021

2009-01-01

Abstract:Validity of training samples is important to regression model,however,outliers in training sample sets are hard to avoid,which will disturb the model accuracy.In order to overcome the influence of outliers to regression model,a new M-estimator based support vector machine(M-SVM) was introduced.It replaced the sum of square residuals in least squares support vector machine(LS-SVM) with objective function of M-estimator.An iterative modeling algorithm of the M-SVM was proposed.The M-SVM was used in a low-dimension data regression problem with outliers and in the NIR spectral analysis of gasoline research octane number.Experimental results showed that the M-SVM was more robust and accurate than other SVMs.
An Unbalanced Dataset Classification Approach Based On V-Support Vector Machine

Yinggang Zhao,Qinming He

DOI: https://doi.org/10.1109/WCICA.2006.1714061

2006-01-01

Abstract:Support Vector Machine (SVM) has been extensively studied and has shown remarkable success in many applications. However, when faced with unbalanced datasets, the SVM can not get ideal classification result and even in some cases the classification ability was very bad and unaccepted. The nu-Support Vector Machine (nu-SVM) is a new formulation of the regular SVM, and its parameter nu has intuitive meanings compared with C (the penalty constant in SVM). By investigating the relation between SVM and nu-SVM, we gave an equation between nu and C, meanwhile we analyzed the factor behind the classification failure of SVM on unbalanced dataset. Then a classification algorithm based on nu-SVM was addressed to overcome this inconvenience. Experimental results show the effectiveness of the proposed algorithm.
Support Vector Machines-Based Method for Machining Error Prediction Modeling

JIANG Liang,LIU Jian,PAN Shuang-xia

DOI: https://doi.org/10.3969/j.issn.1001-2265.2005.08.005

2005-01-01

Abstract:Error compensation is an effective approach to enhance machining precision, and error modeling is the key factor for the compensation. Firstly, the present machining error prediction technologies and their disadvantages were analyzed and compared. Then support vector machines(SVM)-based method of machining error modeling and prediction was put forward. Also some problems in real application of the method were analyzed and summarized. Finally, a real application example and comparison with other modeling methods was given, which indicates that the proposed method has an excellent prediction performance and is a new feasible approach for machining error prediction.
Statistical Mechanics of Support Vector Networks

Rainer Dietrich,Manfred Opper,Haim Sompolinsky

DOI: https://doi.org/10.1103/PhysRevLett.82.2975

1999-02-25

Abstract:Using methods of Statistical Physics, we investigate the generalization performance of support vector machines (SVMs), which have been recently introduced as a general alternative to neural networks. For nonlinear classification rules, the generalization error saturates on a plateau, when the number of examples is too small to properly estimate the coefficients of the nonlinear part. When trained on simple rules, we find that SVMs overfit only weakly. The performance of SVMs is strongly enhanced, when the distribution of the inputs has a gap in feature space.

Disordered Systems and Neural Networks
Fuzzy Support Vector Machine For Regression Estimation

Zh Sun,Yx Sun

DOI: https://doi.org/10.1109/ICSMC.2003.1244404

2003-01-01

Abstract:In recently the theory of support vector machine (SVM) as a tool of pattern classification and regression estimation draw much attention on this field. In this method it maps the input data into a high dimensional characteristic space in which it constructs an optimal separating hyperplane. In many applications it has provided high generalization ability. In this paper we provide the fuzzy SVM for regression estimation problem. In this method it combines the fuzzy logic with the generalize SVM to construct multi-layer SVM. The first layer is the fuzzification process. We apply a fuzzy membership to each data point of SVM and reformulate the SVM such that different input points can make different contributions to the learning of regression function. The second layer is the generalize SVM which make the kernel function may not satisfy the Mercer's condition. The proposed method enhances the SVM in reducing the effect of outliers and noises in the application of data fitting.
Cost Sensitive Support Vector Machines

ZHENG En-hui,LI Ping,SONG Zhi-huan

DOI: https://doi.org/10.3321/j.issn:1001-0920.2006.04.025

2006-01-01

Abstract:Classical methods of designing classifier generally pursue more highly accuracy based on the assumption that all misclassifications have the same cost and the sample number of each class is approximately equal.However,the assumption is not valid in some real applications such as fraud detection and medical diagnosis,so that classification algorithms without taking different misclassification cost into account do not perform well.Based on standard support vector machines(SVM),the algorithm of cost-sensitive SVM(CS-SVM) is proposed by integrating misclassification cost of each sample into standard SVM.Experimental results show that CS-SVM is effective.
Asymmetrical support vector machine based on moving optimal separating hyperplane

Lue Hongsheng,He Jianmin,Hu Xiaoping,Wang Jian

DOI: https://doi.org/10.1109/WCICA.2006.1712815

2006-01-01

Abstract:Aiming at the problem about classifying two samples, support vector machine (SVM) put forward by Vapnik didn't think over the difference of two classes of classification error, so a new method, asymmetrical support vector machine (A-SVM), is given. The optimal separating hyperplane was deviated from the optimal support hyperplane of some kind of sample set by parallel moving the optimal separating hyperplane, and then this kind of sample set could be recognized with higher accurate ratio. Example result shows that A-SVM is similar to SVM for the total recognizing performance of both learning and testing. However, A-SVM is better than SVM when separating the kind of sample set. © 2006 IEEE.
Volumetric Error Prediction and Compensation of NC Machine Tool Based on Least Square Support Vector Machine

Zhenya He,Xinhua Yao,Jianzhong Fu,Zichen Chen

DOI: https://doi.org/10.1166/asl.2011.1660

2011-01-01

Advanced Science Letters

Abstract:A novel method based on Least Square Support Vector Machine (LS-SVM) to predict the volumetric errors of NC machine tools is presented. Using the laser vector diagonal step measurement, the volumetric errors of NC machine tools were measured. Then LS-SVM was used to establish the predictive model of volumetric errors. Based on structural risk minimization, least linear system was used as loss-function, and grid search method was adopted to optimize the LS-SVM parameters, finally the volumetric error model was obtained for prediction and compensation. The experiment results show that the LS-SVM model of volumetric errors is more precise than the Artificial Neural Networks (ANN) model. And after compensation, the accuracy of the machine is improved more than 90.38%. Hence, the LS-SVM volumetric error model is feasible and effective to enhance the performance of NC machine tools.
One new family of smooth semi-supervised support vector classifier based on Fourier series approximation technique

En Wang,Yang Mei

DOI: https://doi.org/10.1007/s10044-024-01365-x

IF: 2.307

2024-11-21

Pattern Analysis and Applications

Abstract:This article presents a new family of smooth semi-supervised support vector machines based on Fourier series approximation technique (FS-S 4 VMs) for classification. The semi-supervised support vector machine (S 3 VM) is the powerful tool for coping with quantities of unlabeled data in the real world. However, the symmetric hinge loss function of S 3 VM is not smooth. It will decrease the classification accuracy and endure heavy burden calculation. To deal with this problem, the Fourier series approximation smooth technique for replacing the non-smooth item have been investigated. One novel family of FS-S 4 VMs classifier is derived. Thus, one fast SAGA algorithm for solving non-convex FS-S 4 VMs can be utilized to decrease the computing scale. The nonlinear case and convergence analysis are presents as well. To attest how the new FS-S 4 VMs can be implemented into practice, experiments on synthetic and real data sets are accessed. In the final, theoretical explanation and simulation comparisons illustrate the FS-S 4 VMs enhance the robustness of S 3 VM, and increase the classification accuracy at about 1%–3% than other smooth techniques.

computer science, artificial intelligence
Generalization Performance Analysis of M-SVMs

Yan Zhigang,Du Peijun

DOI: https://doi.org/10.3969/j.issn.1004-9037.2009.04.013

2009-01-01

Abstract:To analyze the generalization performances of multi-category support vector machines(M-SVMs),the M-SVMs algorithms are used to deduce and summary the theoretical generalization error bounds of M-SVMs.The research results show the ECOC SVMs performance can be improved by constructing suitable codes to a fixed sample space,and H-SVMs performance can be improved by constructing a suitable tree structure to increase M-SVMs performance.Other M-SVMs performances are limited to the sample space.The conclusions provide a theoretical basis for improving M-SVMs performances.
Asymmetric Support Vector Machine For The Classification Problem With Asymmetric Cost Of Misclassification

Zhongsheng Hua,Xuemei Zhang,Xiaoyan Xu

2010-01-01

Abstract:Classification algorithms are usually developed under the assumption that costs of different types of misclassification are equal. In many real world classification problems, however, costs of different types of misclassification are asymmetric, and one type of misclassification is more serious than the other. In this paper, we propose an asymmetric support vector machine (ASVM) for the binary classification problem with asymmetric cost of misclassification. Through introducing some adjustable parameters into the model of the conventional SVM, we establish an explicit connection between the expected cost of misclassification (ECM) and the adjustable parameters. The proposed ASVM effectively decreases ECM through minimizing an upper bound of ECM. Computational results verify the efficacy of ASVM compared with the conventional SVM and the method proposed in the previous literature.
A sparse semismooth Newton based augmented Lagrangian method for large-scale support vector machines

Dunbiao Niu,Chengjing Wang,Peipei Tang,Qingsong Wang,Enbin Song

DOI: https://doi.org/10.48550/arXiv.1910.01312

2021-02-03

Abstract:Support vector machines (SVMs) are successful modeling and prediction tools with a variety of applications. Previous work has demonstrated the superiority of the SVMs in dealing with the high dimensional, low sample size problems. However, the numerical difficulties of the SVMs will become severe with the increase of the sample size. Although there exist many solvers for the SVMs, only few of them are designed by exploiting the special structures of the SVMs. In this paper, we propose a highly efficient sparse semismooth Newton based augmented Lagrangian method for solving a large-scale convex quadratic programming problem with a linear equality constraint and a simple box constraint, which is generated from the dual problems of the SVMs. By leveraging the primal-dual error bound result, the fast local convergence rate of the augmented Lagrangian method can be guaranteed. Furthermore, by exploiting the second-order sparsity of the problem when using the semismooth Newton method,the algorithm can efficiently solve the aforementioned difficult problems. Finally, numerical comparisons demonstrate that the proposed algorithm outperforms the current state-of-the-art solvers for the large-scale SVMs.

Optimization and Control,Machine Learning,Numerical Analysis,Computation
Linear Classification of data with Support Vector Machines and Generalized Support Vector Machines

Xiaomin Qi,Sergei Silvestrov,Talat Nazir

DOI: https://doi.org/10.1063/1.4972718

2016-05-31

Abstract:In this paper, we study the support vector machine and introduced the notion of generalized support vector machine for classification of data. We show that the problem of generalized support vector machine is equivalent to the problem of generalized variational inequality and establish various results for the existence of solutions. Moreover, we provide various examples to support our results.

Machine Learning
Minimal Support Vector Machine

Shuai Zheng,Chris Ding

DOI: https://doi.org/10.48550/arXiv.1804.02370

2018-04-07

Abstract:Support Vector Machine (SVM) is an efficient classification approach, which finds a hyperplane to separate data from different classes. This hyperplane is determined by support vectors. In existing SVM formulations, the objective function uses L2 norm or L1 norm on slack variables. The number of support vectors is a measure of generalization errors. In this work, we propose a Minimal SVM, which uses L0.5 norm on slack variables. The result model further reduces the number of support vectors and increases the classification performance.

Machine Learning
An Efficient Algorithm for a Class of Large-Scale Support Vector Machines Exploiting Hidden Sparsity.

Dunbiao Niu,Chengjing Wang,Peipei Tang,Qingsong Wang,Enbin Song

DOI: https://doi.org/10.1109/tsp.2022.3221837

IF: 4.875

2022-01-01

IEEE Transactions on Signal Processing

Abstract:Support vector machines (SVMs) are successful supervised learning models that analyze data for classification and regression. Previous work has demonstrated the superiority of the SVMs in dealing with the high dimensional, low sample size problems. However, the increase of the sample size brings great challenges to accurately and efficiently solve the large-scale SVMs, especially for the nonlinear kernel SVMs, which may lead to huge computational costs and unaffordable storage burden. In this paper, we propose a highly efficient sparse semismooth Newton (SsN) based augmented Lagrangian (AL) method for solving a class of large-scale SVMs that can be formulated as a convex quadratic programming problem with a linear equality constraint and a simple box constraint. The asymptotic superlinear convergence rate of both the primal and the dual iteration sequences generated by the AL method is guaranteed due to the piecewise linear-quadratic structure of the problem. Furthermore, we reveal the close connection between the number of support vectors and the sparse structure of the generalized Jacobian for the inner subproblem of the AL method. By exploiting this hidden sparsity, the inner subproblem can be solved by the SsN method efficiently and accurately, which greatly reduces the storage burden and computational costs. In particular, for the nonlinear kernel SVMs, since the sparse structure may not manifest in the early iterations of the AL method, we solve a linear kernel SVM approximated by the random Fourier features method to produce a good initial point, and then transfer to solve the original problem. Numerical experiments demonstrate that the proposed algorithm outperforms the current state-of-the-art solvers for the large-scale SVMs.
General Vector Machine

Hong Zhao

DOI: https://doi.org/10.48550/arXiv.1602.03950

2016-02-12

Abstract:The support vector machine (SVM) is an important class of learning machines for function approach, pattern recognition, and time-serious prediction, etc. It maps samples into the feature space by so-called support vectors of selected samples, and then feature vectors are separated by maximum margin hyperplane. The present paper presents the general vector machine (GVM) to replace the SVM. The support vectors are replaced by general project vectors selected from the usual vector space, and a Monte Carlo (MC) algorithm is developed to find the general vectors. The general project vectors improves the feature-extraction ability, and the MC algorithm can control the width of the separation margin of the hyperplane. By controlling the separation margin, we show that the maximum margin hyperplane can usually induce the overlearning, and the best learning machine is achieved with a proper separation margin. Applications in function approach, pattern recognition, and classification indicate that the developed method is very successful, particularly for small-set training problems. Additionally, our algorithm may induce some particular applications, such as for the transductive inference.

Machine Learning

Non-symmetric Support Vector Machines

Flatheaded support vector machine for regression

A Novel Robust Smooth Support Vector Machine

The generalization error of the symmetric and scaled support vector machines.

Weighted Support Vector Machines for Regression and Its Application

M-estimator Based Support Vector Machine and Its Application

An Unbalanced Dataset Classification Approach Based On V-Support Vector Machine

Support Vector Machines-Based Method for Machining Error Prediction Modeling

Statistical Mechanics of Support Vector Networks

Fuzzy Support Vector Machine For Regression Estimation

Cost Sensitive Support Vector Machines

Asymmetrical support vector machine based on moving optimal separating hyperplane

Volumetric Error Prediction and Compensation of NC Machine Tool Based on Least Square Support Vector Machine

One new family of smooth semi-supervised support vector classifier based on Fourier series approximation technique

Generalization Performance Analysis of M-SVMs

Asymmetric Support Vector Machine For The Classification Problem With Asymmetric Cost Of Misclassification

A sparse semismooth Newton based augmented Lagrangian method for large-scale support vector machines

Linear Classification of data with Support Vector Machines and Generalized Support Vector Machines

Minimal Support Vector Machine

An Efficient Algorithm for a Class of Large-Scale Support Vector Machines Exploiting Hidden Sparsity.

General Vector Machine