Abstract:Building highly non-linear and non-parametric models is central to several state-of-the-art machine learning systems. Kernel methods form an important class of techniques that induce a reproducing kernel Hilbert space (RKHS) for inferring non-linear models through the construction of similarity functions from data. These methods are particularly preferred in cases where the training data sizes are limited and when prior knowledge of the data similarities is available. Despite their usefulness, they are limited by the computational complexity and their inability to support end-to-end learning with a task-specific objective. On the other hand, deep neural networks have become the de facto solution for end-to-end inference in several learning paradigms. In this article, we explore the idea of using deep architectures to perform kernel machine optimization, for both computational efficiency and end-to-end inferencing. To this end, we develop the DKMO (Deep Kernel Machine Optimization) framework, that creates an ensemble of dense embeddings using Nystrom kernel approximations and utilizes deep learning to generate task-specific representations through the fusion of the embeddings. Intuitively, the filters of the network are trained to fuse information from an ensemble of linear subspaces in the RKHS. Furthermore, we introduce the kernel dropout regularization to enable improved training convergence. Finally, we extend this framework to the multiple kernel case, by coupling a global fusion layer with pre-trained deep kernel machines for each of the constituent kernels. Using case studies with limited training data, and lack of explicit feature sources, we demonstrate the effectiveness of our framework over conventional model inferencing techniques.

Data-Dependent Kernel Function Based Kernel Optimization Algorithm

A Kernel Function Optimization And Selection Algorithm Based On Cost Function Maximization

The Optimization for Parameter of Kernel Function with GA

Generalized Convexity-Based Inexact Projection Method for Multiple Kernel Learning

Learning with the Optimized Data-Dependent Kernel

Kernel optimization-based discriminant analysis for face recognition

A Gradient-based Kernel Optimization Approach for Parabolic Distributed Parameter Control Systems

Optimizing the Data-Dependent Kernel under a Unified Kernel Optimization Framework

Kernel Parameter Optimization for KFDA Based on the Maximum Margin Criterion.

Kernel Parameter Optimization of Kernel-based LDA Methods

Kernel-Optimized Based Machine for Image Recognition

Optimizing Kernel Function with Applications to Kernel Principal Analysis and Locality Preserving Projection for Feature Extraction.

Bi-criterion Optimization for Kernel Parameters Selection

Maximum Margin Based Semi-supervised Spectral Kernel Learning.

Learning a Data-Dependent Kernel Function for KPCA-based Nonlinear Process Monitoring

Optimizing Kernel Machines using Deep Learning

An Adaptive Approach to Learning Optimal Neighborhood Kernels.

Optimizing the Gaussian Kernel Function with the Formulated Kernel Target Alignment Criterion for Two-Class Pattern Classification.

Learning Data-adaptive Nonparametric Kernels

Adaptive On-Line Optimizing the Gaussian Kernel for Classification Based on the Kernel Target Alignment

A Kernel Optimization Method Based on the Localized Kernel Fisher Criterion.