Abstract:The recently proposed "generalized min-max" (GMM) kernel can be efficiently linearized, with direct applications in large-scale statistical learning and fast near neighbor search. The linearized GMM kernel was extensively compared in with linearized radial basis function (RBF) kernel. On a large number of classification tasks, the tuning-free GMM kernel performs (surprisingly) well compared to the best-tuned RBF kernel. Nevertheless, one would naturally expect that the GMM kernel ought to be further improved if we introduce tuning parameters. In this paper, we study three simple constructions of tunable GMM kernels: (i) the exponentiated-GMM (or eGMM) kernel, (ii) the powered-GMM (or pGMM) kernel, and (iii) the exponentiated-powered-GMM (epGMM) kernel. The pGMM kernel can still be efficiently linearized by modifying the original hashing procedure for the GMM kernel. On about 60 publicly available classification datasets, we verify that the proposed tunable GMM kernels typically improve over the original GMM kernel. On some datasets, the improvements can be astonishingly significant. For example, on 11 popular datasets which were used for testing deep learning algorithms and tree methods, our experiments show that the proposed tunable GMM kernels are strong competitors to trees and deep nets. The previous studies developed tree methods including "abc-robust-logitboost" and demonstrated the excellent performance on those 11 datasets (and other datasets), by establishing the second-order tree-split formula and new derivatives for multi-class logistic loss. Compared to tree methods like "abc-robust-logitboost" (which are slow and need substantial model sizes), the tunable GMM kernels produce largely comparable results.

Training Discriminative HMM by Optimal Allocation of Gaussian Kernels

Distance-Based Classifier Via the Kernel Trick.

Discriminative training of GMM-HMM acoustic model by RPCL learning

Discriminative training of GMM-HMM acoustic model by RPCL type Bayesian Ying-Yang harmony learning

A New Minimum Divergence Approach To Discriminative Training

The application of discriminative training techniques in LID system fusion

Exploring More Representative States of Hidden Markov Model in Optical Character Recognition: A Clustering-Based Model Pre-Training Approach

Discriminative Dynamic Gaussian Mixture Selection with Enhanced Robustness and Performance for Multi-Accent Speech Recognition

Performance of Discriminative HMM Training in Noise.

Kernel Generalized Nonlinear Discriminant Analysis Algorithm For Pattern Recognition

Boosting Gaussian Mixture Models Via Discriminant Analysis

Constructing the discriminative kernels using GMM for text-independent speaker identification

Regularization of Hidden Markov Models Embedded into Reproducing Kernel Hilbert Space

Several Tunable GMM Kernels

Improved kernel direct Fisher discriminant analysis and face recognition

Training Radial Basis Function Classifiers with Gaussian Kernel Clustering and Fuzzy Decision Technique

Tunable GMM Kernels

A new hybrid discriminative/generative model using the full-covariance multivariate generalized Gaussian mixture models

Discriminative Training and Explicit Duration Modeling for HMM-based Automatic Segmentation

Minimum Divergence Based Discriminative Training.

Log-Likelihood Kernels Based on Adapted GMMs for Speaker Verification