Abstract:The canonical support vector machines (SVMs) are based on a single kernel, recent publications have shown that using multiple kernels instead of a single one can enhance interpretability of the decision function and promote classification accuracy. However, most of existing approaches mainly reformulate the multiple kernel learning as a saddle point optimization problem which concentrates on solving the dual. In this paper, we show that the multiple kernel learning (MKL) problem can be reformulated as a BiConvex optimization and can also be solved in the primal. While the saddle point method still lacks convergence results, our proposed method exhibits strong optimization convergence properties. To solve the MKL problem, a two-stage algorithm that optimizes canonical SVMs and kernel weights alternately is proposed. Since standard Newton and gradient methods are too time-consuming, we employ the truncated-Newton method to optimize the canonical SVMs. The Hessian matrix need not be stored explicitly, and the Newton direction can be computed using several Preconditioned Conjugate Gradient steps on the Hessian operator equation, the algorithm is shown more efficient than the current primal approaches in this MKL setting. Furthermore, we use the Nesterov’s optimal gradient method to optimize the kernel weights. One remarkable advantage of solving in the primal is that it achieves much faster convergence rate than solving in the dual and does not require a two-stage algorithm even for the single kernel LapSVM. Introducing the Laplacian regularizer, we also extend our primal method to semi-supervised scenario. Extensive experiments on some UCI benchmarks have shown that the proposed algorithm converges rapidly and achieves competitive accuracy.

Nonlinear Combination of Multiple Kernels for Support Vector Machines

Generalized Convexity-Based Inexact Projection Method for Multiple Kernel Learning

An Improved Domain Multiple Kernel Support Vector Machine

Efficient Multiple Kernel Learning Algorithms Using Low-Rank Representation.

More Efficient Sparse Multi-kernel Based Least Square Support Vector Machine

Joint Learning of Distance Metric and Kernel Classifier via Multiple Kernel Learning.

A Primal Method for Multiple Kernel Learning

Kernel-Free Nonlinear Support Vector Machines for Multiview Binary Classification Problems

Multiple Locally Linear Kernel Machines

A Multiple Kernel Learning Model Based on P-Norm

Laplacian Support Vector Machines with Multi-Kernel Learning

Multiple Kernel Subspace Learning for Clustering and Classification

Traffic Incident Detection Using Multiple-Kernel Support Vector Machine

Evolutionary Multiple Kernels Design for Support Vector Machines

Support Vector Machine Improvement with Kemel-Based Nonlinear Representor

Mixtures of kernels for SVM modeling

Multiple graph kernel learning based on GMDH-type neural network

Localized Multiple Kernel Learning Via Sample-Wise Alternating Optimization

Multikernel semiparametric linear programming support vector regression

Nonlinear Kernel Support Vector Machine with 0-1 Soft Margin Loss

Facilitating the Applications of Support Vector Machine by Using a New Kernel