Abstract:Kernel methods have been successfully applied in various applications. To succeed in these applications, it is crucial to learn a good kernel representation, whose objective is to reveal the data similarity precisely. In this paper, we address the problem of multiple kernel learning (MKL), searching for the optimal kernel combination weights through maximizing a generalized performance measure. Most MKL methods employ the L(1)-norm simplex constraints on the kernel combination weights, which therefore involve a sparse but non-smooth solution for the kernel weights. Despite the success of their efficiency, they tend to discard informative complementary or orthogonal base kernels and yield degenerated generalization performance. Alternatively, imposing the L(p)-norm (p > 1) constraint on the kernel weights will keep all the information in the base kernels. This leads to non-sparse solutions and brings the risk of being sensitive to noise and incorporating redundant information. To tackle these problems, we propose a generalized MKL (GMKL) model by introducing an elastic-net-type constraint on the kernel weights. More specifically, it is an MKL model with a constraint on a linear combination of the L(1)-norm and the squared L(2)-norm on the kernel weights to seek the optimal kernel combination weights. Therefore, previous MKL problems based on the L(1)-norm or the L(2)-norm constraints can be regarded as special cases. Furthermore, our GMKL enjoys the favorable sparsity property on the solution and also facilitates the grouping effect. Moreover, the optimization of our GMKL is a convex optimization problem, where a local solution is the global optimal solution. We further derive a level method to efficiently solve the optimization problem. A series of experiments on both synthetic and real-world datasets have been conducted to show the effectiveness and efficiency of our GMKL.

MPGL: an Efficient Matching Pursuit Method for Generalized LASSO

GLM+: An Efficient System for Generalized Linear Models

Smoothing composite proximal gradient algorithm for sparse group Lasso problems with nonsmooth loss functions

Efficient Generalized Fused Lasso and Its Applications.

Overlapping Group Lasso for High-Dimensional Generalized Linear Models

An Algorithm for the Estimation of Regularization Paths of Generalized Linear Models with Group LASSO Penalty

Geometry-dependent matching pursuit: a transition phase for convergence on linear regression and LASSO

Efficient Projected Gradient Methods for L 0 Constrained Optimization

An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems

Efficient Lasso Training from a Geometrical Perspective.

Generalized Orthogonal Matching Pursuit with Singular Value Decomposition

A Fast Method for Lasso and Logistic Lasso

Sparse Gradient Pursuit For Robust Visual Analysis

GGLasso -- a Python package for General Graphical Lasso computation

An Accelerating Outer Space Algorithm for Globally Solving Generalized Linear Multiplicative Problems

A new linearization method for generalized linear multiplicative programming

Efficient sparse generalized multiple kernel learning

A re-examination to the SCoTLASS problems for SPCA and two projection-based methods for them

Exploiting regularity in sparse Generalized Linear Models

A Generalization of Linearized Alternating Direction Method of Multipliers for Solving Two-Block Separable Convex Programming

Efficient Inexact Proximal Gradient Algorithms for Structured Sparsity-Inducing Norm.