Abstract:The canonical support vector machines (SVMs) are based on a single kernel, recent publications have shown that using multiple kernels instead of a single one can enhance interpretability of the decision function and promote classification accuracy. However, most of existing approaches mainly reformulate the multiple kernel learning as a saddle point optimization problem which concentrates on solving the dual. In this paper, we show that the multiple kernel learning (MKL) problem can be reformulated as a BiConvex optimization and can also be solved in the primal. While the saddle point method still lacks convergence results, our proposed method exhibits strong optimization convergence properties. To solve the MKL problem, a two-stage algorithm that optimizes canonical SVMs and kernel weights alternately is proposed. Since standard Newton and gradient methods are too time-consuming, we employ the truncated-Newton method to optimize the canonical SVMs. The Hessian matrix need not be stored explicitly, and the Newton direction can be computed using several Preconditioned Conjugate Gradient steps on the Hessian operator equation, the algorithm is shown more efficient than the current primal approaches in this MKL setting. Furthermore, we use the Nesterov’s optimal gradient method to optimize the kernel weights. One remarkable advantage of solving in the primal is that it achieves much faster convergence rate than solving in the dual and does not require a two-stage algorithm even for the single kernel LapSVM. Introducing the Laplacian regularizer, we also extend our primal method to semi-supervised scenario. Extensive experiments on some UCI benchmarks have shown that the proposed algorithm converges rapidly and achieves competitive accuracy.

Two-stage multiple kernel learning with multiclass kernel polarization.

Generalized Convexity-Based Inexact Projection Method for Multiple Kernel Learning

Self-weighted Multiple Kernel Learning for Graph-Based Clustering and Semi-Supervised Classification

CLASS-PAIR-GUIDED MULTIPLE KERNEL LEARNING OF INTEGRATING HETEROGENEOUS FEATURES FOR CLASSIFICATION

Multiclass Kernel Polarization and Its Application to Parameter Selection of RBF Kernel with Multiple Widths

Coupled Multiple Kernel Learning for Supervised Classification.

Parallel multiple kernel learning

Ideal Kernel-Based Multiple Kernel Learning for Spectral-Spatial Classification of Hyperspectral Image.

Parallel Multiple Kernel Learning: a Hybrid Alternating Direction Method of Multipliers

Efficient Multiple Kernel Learning Algorithms Using Low-Rank Representation.

Representative Multiple Kernel Learning for Classification in Hyperspectral Imagery

Multiple Kernel Learning for Label Relation and Class Imbalance in Multi-Label Learning

Multiple Kernel Subspace Learning for Clustering and Classification

Joint Learning of Distance Metric and Kernel Classifier via Multiple Kernel Learning.

A Primal Method for Multiple Kernel Learning

Selective multiple kernel learning for classification with ensemble strategy.

Multiple Kernel Learning Based on Ideal Kernel Optimization for Hyperspectral Imagery Classification

Unsupervised Multiple Kernel Learning

A Novel Multiple Kernel Boosting Method For Hyperspectral Image Classification

Multiple Kernel Learning via Low-Rank Nonnegative Matrix Factorization for Classification of Hyperspectral Imagery

Simultaneous learning coefficient matrix and affinity graph for multiple kernel clustering