Abstract:The canonical support vector machines (SVMs) are based on a single kernel, recent publications have shown that using multiple kernels instead of a single one can enhance interpretability of the decision function and promote classification accuracy. However, most of existing approaches mainly reformulate the multiple kernel learning as a saddle point optimization problem which concentrates on solving the dual. In this paper, we show that the multiple kernel learning (MKL) problem can be reformulated as a BiConvex optimization and can also be solved in the primal. While the saddle point method still lacks convergence results, our proposed method exhibits strong optimization convergence properties. To solve the MKL problem, a two-stage algorithm that optimizes canonical SVMs and kernel weights alternately is proposed. Since standard Newton and gradient methods are too time-consuming, we employ the truncated-Newton method to optimize the canonical SVMs. The Hessian matrix need not be stored explicitly, and the Newton direction can be computed using several Preconditioned Conjugate Gradient steps on the Hessian operator equation, the algorithm is shown more efficient than the current primal approaches in this MKL setting. Furthermore, we use the Nesterov’s optimal gradient method to optimize the kernel weights. One remarkable advantage of solving in the primal is that it achieves much faster convergence rate than solving in the dual and does not require a two-stage algorithm even for the single kernel LapSVM. Introducing the Laplacian regularizer, we also extend our primal method to semi-supervised scenario. Extensive experiments on some UCI benchmarks have shown that the proposed algorithm converges rapidly and achieves competitive accuracy.

Kernel partial least squares based on least squares support vector machine prima l-dual optimization problem

Generalized Convexity-Based Inexact Projection Method for Multiple Kernel Learning

Least squares support vector machine based partially linear model identification

A Kernel Function Optimization And Selection Algorithm Based On Cost Function Maximization

Reduced Least Squares Support Vector Based On Kernel Partial Least Squares And Its Application Research

On-line Weighted Least Squares Kernel Method for Nonlinear Dynamic Modeling

Nonlinear Dynamic Process Monitoring Based on Kernel Partial Least Squares

Study on Kernel Partial Least Squares Based Key Indicator Prediction.

Nonlinear Multivariate Quality Estimation and Prediction Based on Kernel Partial Least Squares

A splitting primal-dual proximity algorithm for solving composite optimization problems

A Primal Method for Multiple Kernel Learning

An efficient Kernel-based matrixized least squares support vector machine

A Kernelizable Primal-Dual Formulation of the Multilinear Singular Value Decomposition

Nonlinear Process Modeling and Optimization Based on Multiway Kernel Partial Least Squares Model

Design of a Multiple Kernel Learning Algorithm for LS-SVM by Convex Programming

Support Vector Machine Based Nonlinear Model Multi-Step-ahead Optimizing Predictive Control

Cost-Constraint Based Multiple Kernel Least Squares Support Vector Machine and Its Application

Sparse multiple kernel for least square support vector regression

Kernel partial least squares regression for functional nonlinear models

A novel multivariate regression approach based on kernel partial least squares with orthogonal signal correction

Least Squares Hidden Space Support Vector Machines