Abstract:Building highly non-linear and non-parametric models is central to several state-of-the-art machine learning systems. Kernel methods form an important class of techniques that induce a reproducing kernel Hilbert space (RKHS) for inferring non-linear models through the construction of similarity functions from data. These methods are particularly preferred in cases where the training data sizes are limited and when prior knowledge of the data similarities is available. Despite their usefulness, they are limited by the computational complexity and their inability to support end-to-end learning with a task-specific objective. On the other hand, deep neural networks have become the de facto solution for end-to-end inference in several learning paradigms. In this article, we explore the idea of using deep architectures to perform kernel machine optimization, for both computational efficiency and end-to-end inferencing. To this end, we develop the DKMO (Deep Kernel Machine Optimization) framework, that creates an ensemble of dense embeddings using Nystrom kernel approximations and utilizes deep learning to generate task-specific representations through the fusion of the embeddings. Intuitively, the filters of the network are trained to fuse information from an ensemble of linear subspaces in the RKHS. Furthermore, we introduce the kernel dropout regularization to enable improved training convergence. Finally, we extend this framework to the multiple kernel case, by coupling a global fusion layer with pre-trained deep kernel machines for each of the constituent kernels. Using case studies with limited training data, and lack of explicit feature sources, we demonstrate the effectiveness of our framework over conventional model inferencing techniques.

Kernel-Optimized Based Machine for Image Recognition

A Kernel Function Optimization And Selection Algorithm Based On Cost Function Maximization

Generalized Convexity-Based Inexact Projection Method for Multiple Kernel Learning

The Optimization for Parameter of Kernel Function with GA

Kernel optimization-based discriminant analysis for face recognition

Optimizing Kernel Machines using Deep Learning

Optimizing Kernel Function with Applications to Kernel Principal Analysis and Locality Preserving Projection for Feature Extraction.

Learning with the Optimized Data-Dependent Kernel

A General Kernelization Framework for Learning Algorithms Based on Kernel PCA.

Data-Dependent Kernel Function Based Kernel Optimization Algorithm

Kernel Optimization Based Semi-Supervised KBDA Scheme for Image Retrieval.

Online Kernel Dictionary Learning for Object Recognition

Self-supervised Adaptive Kernel Nonnegative Matrix Factorization.

Kernelized dual regression incorporating local information for image set classification

Learning a Data-Dependent Kernel Function for KPCA-based Nonlinear Process Monitoring

Kimel: A Kernel Incremental Metalearning Algorithm

Research on Optimization of Image Recognition Algorithm Based on Deep Learning

Research on Optimizing Kernel Discriminant Analysis

High-performance Kernel Machines with Implicit Distributed Optimization and Randomization

Supervised Kernel Descriptors for Visual Recognition

Image Classification Based on KPCA and SVM with Randomized Hyper-parameter Optimization