Abstract:Kernel methods have been successfully applied in various applications. To succeed in these applications, it is crucial to learn a good kernel representation, whose objective is to reveal the data similarity precisely. In this paper, we address the problem of multiple kernel learning (MKL), searching for the optimal kernel combination weights through maximizing a generalized performance measure. Most MKL methods employ the L(1)-norm simplex constraints on the kernel combination weights, which therefore involve a sparse but non-smooth solution for the kernel weights. Despite the success of their efficiency, they tend to discard informative complementary or orthogonal base kernels and yield degenerated generalization performance. Alternatively, imposing the L(p)-norm (p > 1) constraint on the kernel weights will keep all the information in the base kernels. This leads to non-sparse solutions and brings the risk of being sensitive to noise and incorporating redundant information. To tackle these problems, we propose a generalized MKL (GMKL) model by introducing an elastic-net-type constraint on the kernel weights. More specifically, it is an MKL model with a constraint on a linear combination of the L(1)-norm and the squared L(2)-norm on the kernel weights to seek the optimal kernel combination weights. Therefore, previous MKL problems based on the L(1)-norm or the L(2)-norm constraints can be regarded as special cases. Furthermore, our GMKL enjoys the favorable sparsity property on the solution and also facilitates the grouping effect. Moreover, the optimization of our GMKL is a convex optimization problem, where a local solution is the global optimal solution. We further derive a level method to efficiently solve the optimization problem. A series of experiments on both synthetic and real-world datasets have been conducted to show the effectiveness and efficiency of our GMKL.

More Efficient Sparse Multi-kernel Based Least Square Support Vector Machine

Generalized Convexity-Based Inexact Projection Method for Multiple Kernel Learning

Non-parametric Least Square Support Vector Machine

Cost-Constraint Based Multiple Kernel Least Squares Support Vector Machine and Its Application

Sparse multiple kernel for least square support vector regression

Nonlinear Combination of Multiple Kernels for Support Vector Machines

Efficient Multiple Kernel Learning Algorithms Using Low-Rank Representation.

Least Squares Support Vector Machine with Self-Organizing Multiple Kernel Learning and Sparsity.

Design of a Multiple Kernel Learning Algorithm for LS-SVM by Convex Programming

An Efficient Algorithm for a Class of Large-Scale Support Vector Machines Exploiting Hidden Sparsity.

An Improved Domain Multiple Kernel Support Vector Machine

Efficient Sparse Least Squares Support Vector Machines for Regression

Least Squares Support Vector Machine Regression Based on Sparse Samples and Mixture Kernel Learning.

A sparse semismooth Newton based augmented Lagrangian method for large-scale support vector machines

Efficient sparse generalized multiple kernel learning

A Primal Method for Multiple Kernel Learning

An efficient Kernel-based matrixized least squares support vector machine

Laplacian Support Vector Machines with Multi-Kernel Learning

Two-stage sparse multi-kernel optimization classifier method for more accurate and explainable prediction

Spare Simple MKKM with Semi‐infinite Linear Program Optimization

A New Formulation of Sparse Multiple Kernel K$$ K $$‐means Clustering and Its Applications