Abstract:Learning an optimal kernel plays a pivotal role in kernel-based methods. Recently, an approach called optimal neighborhood kernel learning (ONKL) has been proposed, showing promising classification performance. It assumes that the optimal kernel will reside in the neighborhood of a "pre-specified" kernel. Nevertheless, how to specify such a kernel in a principled way remains unclear. To solve this issue, this paper treats the pre-specified kernel as an extra variable and jointly learns it with the optimal neighborhood kernel and the structure parameters of support vector machines. To avoid trivial solutions, we constrain the pre-specified kernel with a parameterized model. We first discuss the characteristics of our approach and in particular highlight its adaptivity. After that, two instantiations are demonstrated by modeling the pre-specified kernel as a common Gaussian radial basis function kernel and a linear combination of a set of base kernels in the way of multiple kernel learning (MKL), respectively. We show that the optimization in our approach is a min-max problem and can be efficiently solved by employing the extended level method and Nesterov's method. Also, we give the probabilistic interpretation for our approach and apply it to explain the existing kernel learning methods, providing another perspective for their commonness and differences. Comprehensive experimental results on 13 UCI data sets and another two real-world data sets show that via the joint learning process, our approach not only adaptively identifies the pre-specified kernel, but also achieves superior classification performance to the original ONKL and the related MKL algorithms.

Optimizing Kernel Function with Applications to Kernel Principal Analysis and Locality Preserving Projection for Feature Extraction.

Preservation of Local Linearity by Neighborhood Subspace Scaling for Solving the Pre-Image Problem.

Generalized Convexity-Based Inexact Projection Method for Multiple Kernel Learning

A Kernel Function Optimization And Selection Algorithm Based On Cost Function Maximization

Kernel optimization-based discriminant analysis for face recognition

Kernel-Optimized Based Machine for Image Recognition

Locality preserving kernel on graphs

A General Kernelization Framework for Learning Algorithms Based on Kernel PCA.

Learning a Data-Dependent Kernel Function for KPCA-based Nonlinear Process Monitoring

Feature Extraction Based on Supervised Kernel Locality Preserving Projection Algorithm and Application to Fault Diagnosis

Sparse Data-Dependent Kernel Principal Component Analysis Based on Least Squares Support Vector Machine for Feature Extraction and Recognition.

Locality Preserving Kernel Method and Its Application

Supervised Kernel Optimized Locality Preserving Projection with Its Application to Face Recognition and Palm Biometrics

Kernel Optimisation for KPCA Based on Gaussianity Estimation.

A Kernel Orthogonal Discriminant Locality Preserving Projections Method

Learning with the Optimized Data-Dependent Kernel

Manifold based kernel optimization for KPCA

A Framework of Quasiconformal Mapping-Based Kernel Machine with Its Application to Hyperspectral Remote Sensing

Robust kernel discriminant analysis and its application to feature extraction and recognition

An Adaptive Approach to Learning Optimal Neighborhood Kernels.

<Bold>An Implementation Framework For Kernel Methods With High-Dimensional Patterns</Bold>