Abstract:Kernel methods have been successfully applied in various applications. To succeed in these applications, it is crucial to learn a good kernel representation, whose objective is to reveal the data similarity precisely. In this paper, we address the problem of multiple kernel learning (MKL), searching for the optimal kernel combination weights through maximizing a generalized performance measure. Most MKL methods employ the L(1)-norm simplex constraints on the kernel combination weights, which therefore involve a sparse but non-smooth solution for the kernel weights. Despite the success of their efficiency, they tend to discard informative complementary or orthogonal base kernels and yield degenerated generalization performance. Alternatively, imposing the L(p)-norm (p > 1) constraint on the kernel weights will keep all the information in the base kernels. This leads to non-sparse solutions and brings the risk of being sensitive to noise and incorporating redundant information. To tackle these problems, we propose a generalized MKL (GMKL) model by introducing an elastic-net-type constraint on the kernel weights. More specifically, it is an MKL model with a constraint on a linear combination of the L(1)-norm and the squared L(2)-norm on the kernel weights to seek the optimal kernel combination weights. Therefore, previous MKL problems based on the L(1)-norm or the L(2)-norm constraints can be regarded as special cases. Furthermore, our GMKL enjoys the favorable sparsity property on the solution and also facilitates the grouping effect. Moreover, the optimization of our GMKL is a convex optimization problem, where a local solution is the global optimal solution. We further derive a level method to efficiently solve the optimization problem. A series of experiments on both synthetic and real-world datasets have been conducted to show the effectiveness and efficiency of our GMKL.

Sparse Multiple Kernel Learning: Minimax Rates with Random Projection

Generalized Convexity-Based Inexact Projection Method for Multiple Kernel Learning

Accelerated Sparse Linear Regression Via Random Projection.

Online Kernel Learning with a Near Optimal Sparsity Bound

An Efficient and Effective Multiple Empirical Kernel Learning Based on Random Projection

Exclusive Sparsity Norm Minimization with Random Groups Via Cone Projection

Kernel Regression with Sparse Metric Learning

Sparse Random Projection for Χ2 Kernel Linearization: Algorithm and Applications to Image Classification

Sparse Learning for Large-Scale and High-Dimensional Data: A Randomized Convex-Concave Optimization Approach

Binary Random Projections with Controllable Sparsity Patterns

Efficient sparse generalized multiple kernel learning

Accumulations of Projections--A Unified Framework for Random Sketches in Kernel Ridge Regression

Nonparametric Testing under Randomized Sketching

Random projection ensemble learning with multiple empirical kernels

Variance Reduced Random Relaxed Projection Method for Constrained Finite-Sum Minimization Problems

Sparsity-Aware Distributed Learning for Gaussian Processes with Linear Multiple Kernel

Robust Locality Preserving Projection Based on Kernel Risk-Sensitive Loss

Sparse Data-Driven Random Projection in Regression for High-Dimensional Data

Generalization Bound and Learning Methods for Data-Driven Projections in Linear Programming

A minimax optimal approach to high-dimensional double sparse linear regression