Abstract:The need for scalable and expressive models in machine learning is paramount, particularly in applications requiring both structural depth and flexibility. Traditional deep learning methods, such as multilayer perceptrons (MLP), offer depth but lack ability to integrate structural characteristics of deep learning architectures with non-parametric flexibility of kernel methods. To address this, deep kernel learning (DKL) was introduced, where inputs to a base kernel are transformed using a deep learning architecture. These kernels can replace standard kernels, allowing both expressive power and scalability. The advent of Kolmogorov-Arnold Networks (KAN) has generated considerable attention and discussion among researchers in scientific domain. In this paper, we introduce a scalable deep kernel using KAN (DKL-KAN) as an effective alternative to DKL using MLP (DKL-MLP). Our approach involves simultaneously optimizing these kernel attributes using marginal likelihood within a Gaussian process framework. We analyze two variants of DKL-KAN for a fair comparison with DKL-MLP: one with same number of neurons and layers as DKL-MLP, and another with approximately same number of trainable parameters. To handle large datasets, we use kernel interpolation for scalable structured Gaussian processes (KISS-GP) for low-dimensional inputs and KISS-GP with product kernels for high-dimensional inputs. The efficacy of DKL-KAN is evaluated in terms of computational training time and test prediction accuracy across a wide range of applications. Additionally, the effectiveness of DKL-KAN is also examined in modeling discontinuities and accurately estimating prediction uncertainty. The results indicate that DKL-KAN outperforms DKL-MLP on datasets with a low number of observations. Conversely, DKL-MLP exhibits better scalability and higher test prediction accuracy on datasets with large number of observations.

Shared Deep Kernel Learning for Dimensionality Reduction.

DMT-EV: an Explainable Deep Network for Dimension Reduction.

Deep Manifold Transformation for Dimension Reduction

Deep Latent-Variable Kernel Learning

Guided Deep Kernel Learning

Deep kernelized dimensionality reducer for multi-modality heterogeneous data

Data-Dependent Kernel Discriminant Analysis for Feature Extraction and Classification

Learning Explicit Deep Representations from Deep Kernel Networks

Deep Dimension Reduction for Supervised Representation Learning

Dimensionality Reduction by Mixed Kernel Canonical Correlation Analysis

C2DNDA: A Deep Framework for Nonlinear Dimensionality Reduction

DKL-KAN: Scalable Deep Kernel Learning using Kolmogorov-Arnold Networks

A Supervised Method For Nonlinear Dimensionality Reduction With Gplvm

An Improved Deep Kernel Partial Least Squares and Its Application to Industrial Data Modeling

Kernel-Induced Space Selection Approach to LPKHDA Dimensional Reduction Algorithm

Physics Informed Deep Kernel Learning

Deep Generative LDA.

Deep Kernel Learning Networks with Multiple Learning Paths

Unsupervised 2D Dimensionality Reduction with Adaptive Structure Learning

Nonlinear Dimensionality Reduction of Data by Deep Distributed Random Samplings.

A Deep Learning Approach To Multiple Kernel Fusion