Abstract:Kernel functions in support vector machines (SVM) are needed to assess the similarity of input samples in order to classify these samples, for instance. Besides standard kernels such as Gaussian (i.e., radial basis function, RBF) or polynomial kernels, there are also specific kernels tailored to consider structure in the data for similarity assessment. In this paper, we will capture structure in data by means of probabilistic mixture density models, for example Gaussian mixtures in the case of real-valued input spaces. From the distance measures that are inherently contained in these models, e.g., Mahalanobis distances in the case of Gaussian mixtures, we derive a new kernel, the responsibility weighted Mahalanobis (RWM) kernel. Basically, this kernel emphasizes the influence of model components from which any two samples that are compared are assumed to originate (that is, the “responsible” model components). We will see that this kernel outperforms the RBF kernel and other kernels capturing structure in data (such as the LAP kernel in Laplacian SVM) in many applications where partially labeled data are available, i.e., for semi-supervised training of SVM. Other key advantages are that the RWM kernel can easily be used with standard SVM implementations and training algorithms such as sequential minimal optimization, and heuristics known for the parametrization of RBF kernels in a C-SVM can easily be transferred to this new kernel. Properties of the RWM kernel are demonstrated with 20 benchmark data sets and an increasing percentage of labeled samples in the training data.

Generalization Performance of Gaussian Kernels SVMC Based on Markov Sampling.

The Generalization Ability of SVM Classification Based on Markov Sampling

The Learning Performance of Support Vector Machine Classification Based on Markov Sampling

The Generalization Performance of Regularized Regression Algorithms Based on Markov Sampling

K-Times Markov Sampling for SVMC

Generalization Capacity of Multi-class SVM Based on Markovian Resampling

Learning performance of regularized regression with multiscale kernels based on Markov observations

Learning from uniformly ergodic Markov chains

Generalization Performance of Least-Square Regularized Regression Algorithm with Markov Chain Samples

Learning Performance of Weighted Distributed Learning With Support Vector Machines

SVM-Boosting based on Markov resampling: Theory and algorithm

Learning with multiple Gaussian distance kernels for time series classification

OAA-SVM-MS: A fast and efficient multi-class classification algorithm

Generalization Bounds of ERM Algorithm with Markov Chain Samples

MK-MSMCR: An efficient multiple kernel approach to multi-class classification

A Learning Algorithm for Enhancing the Generalization Ability of Support Vector Machines

The Study Of An Improved Relevant Vector Machine Mud Based On Perfect Sampling

Efficient Model Selection for Support Vector Machine with Gaussian Kernel Function

The responsibility weighted Mahalanobis kernel for semi-supervised training of support vector machines for classification

Support vector machine algorithm based on Gaussian mixture model and spatial ambiguity

Learning Distance Metric for Support Vector Machine: A Multiple Kernel Learning Approach.