Incremental batch learning with support vector machines

Yangguang Liu,Qinming He,Qi Chen

DOI: https://doi.org/10.1109/wcica.2004.1340997

2004-01-01

Abstract:Support Vector Machines (SVMs) have become a popular tool for learning with large amounts of high dimensional data. But it may sometimes be preferable to learn incrementally from previous SVM results, as SVMs which involve the solution of a quadratic programming problem suffer from the problem of large memory requirement and CPU time when trained in batch mode on large data sets. And the SVMs may be used in online learning setting. We proposed an approach for incremental batch learning with Support Vector Machines for classification and regression, and define the normal solution of the incremental batch learning with SVMs as the solution minimizing a given positive-definite quadratic form in the coordinates of the difference vector between the normal vectors at the (k-1)-th and k-th incremental batch step and discuss the relation to standard SVM. It is shown that concept learned at last step will not change if new data satisfy separable condition and empirical evidence is given to prove that this approach can effectively deal with changes in the target concept that are results of the incremental learning setting according to three evaluation criteria: stability, improvement and recoverability.