Abstract:Kernel methods and deep learning are two of the most currently remarkable machine learning techniques that have achieved great success in many applications. Kernel methods are powerful tools to capture nonlinear patterns behind data. They implicitly learn high (even infinite) dimensional nonlinear features in the reproducing kernel Hilbert space (RKHS) while making the computation tractable by leveraging the kernel trick. It is commonly agreed that the success of kernel methods is very much dependent on the choice of kernel. Multiple kernel learning (MKL) is one possible scheme that performs kernel combination and selection for a variety of learning tasks, such as classification, clustering, and dimensionality reduction. Deep learning models project input data through several layers of nonlinearity and learn different levels of abstraction. The composition of multiple layers of nonlinear functions can approximate a rich set of naturally occurring input-output dependencies. To bridge kernel methods and deep learning, deep kernel learning has been proven to be an effective method to learn complex feature representations by combining the nonparametric flexibility of kernel methods with the structural properties of deep learning. This article presents a comprehensive overview of the state-of-the-art approaches that bridge the MKL and deep learning techniques. Specifically, we systematically review the typical hybrid models, training techniques, and their theoretical and practical benefits, followed by remaining challenges and future directions. We hope that our perspectives and discussions serve as valuable references for new practitioners and theoreticians seeking to innovate in the applications of the approaches incorporating the advantages of both paradigms and exploring new synergies.

Kernel-based learning machines research overview

Generalized Convexity-Based Inexact Projection Method for Multiple Kernel Learning

Extreme Learning Machine with Multiple Kernels

A Kernel Function Optimization And Selection Algorithm Based On Cost Function Maximization

An Overview of Kernel Alignment and Its Applications

A General Kernelization Framework for Learning Algorithms Based on Kernel PCA.

Kernel machines with two layers and multiple kernel learning

Bridging deep and multiple kernel learning: A review

A Semi-supervised Kernel Learning Method Based on Label Propagation

SVM Classification:Its Contents and Challenges

Optimizing Kernel Machines using Deep Learning

A Novel Kernel-Based Maximum a Posteriori Classification Method

On Kernel Method-Based Connectionist Models and Supervised Deep Learning Without Backpropagation

A Learnable Kernel Machine for Short-Term Load Forecasting

A Unified Kernel for Neural Network Learning

Kernel Mse Algorithm: A Unified Framework for Kfd, Ls-Svm and Krr

Adaptive Kernel Principal Component Analysis with Unsupervised Learning of Kernels

A Kernel Machine Learning for Inverse Source and Scattering Problems

Kernel Methods in Genomics and Computational Biology

A Review of Kernel Methods for Feature Extraction in Nonlinear Process Monitoring

Research on Machine Learning with Algorithms and Development