Abstract:The recently proposed Kolmogorov-Arnold Networks (KANs) offer enhanced interpretability and greater model expressiveness. However, KANs also present challenges related to privacy leakage during inference. Homomorphic encryption (HE) facilitates privacy-preserving inference for deep learning models, enabling resource-limited users to benefit from deep learning services while ensuring data security. Yet, the complex structure of KANs, incorporating nonlinear elements like the SiLU activation function and B-spline functions, renders existing privacy-preserving inference techniques inadequate. To address this issue, we propose an accurate and efficient privacy-preserving inference scheme tailored for KANs. Our approach introduces a task-specific polynomial approximation for the SiLU activation function, dynamically adjusting the approximation range to ensure high accuracy on real-world datasets. Additionally, we develop an efficient method for computing B-spline functions within the HE domain, leveraging techniques such as repeat packing, lazy combination, and comparison functions. We evaluate the effectiveness of our privacy-preserving KAN inference scheme on both symbolic formula evaluation and image classification. The experimental results show that our model achieves accuracy comparable to plaintext KANs across various datasets and outperforms plaintext MLPs. Additionally, on the CIFAR-10 dataset, our inference latency achieves over 7 times speedup compared to the naive method.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the challenges faced by Kolmogorov - Arnold Networks (KANs) in the privacy - protected inference process. Specifically, KANs provide stronger interpretability and model - expressing ability, but there is a risk of privacy leakage during the inference process. Although the existing Homomorphic Encryption (HE) technology can achieve privacy - protected inference of deep - learning models, due to the complex structure of KANs, which contains non - linear elements such as SiLU activation functions and B - spline functions, it is difficult for the existing privacy - protected inference techniques to be directly applied to KANs. To address these challenges, the paper proposes an efficient privacy - protected inference scheme for KANs. The following are the main contributions of this scheme: 1. **Approximation of SiLU Activation Function**: - Since the SiLU activation function has no direct relation to the sign function, its approximation is more challenging than that of ReLU. The paper proposes a task - specific polynomial approximation method, which dynamically adjusts the approximation range and uses the weighted least - squares method to fit the activation function. This method performs well on actual datasets, and the formula is as follows: \[ \min\left(\sum_{i = 1}^{n}w_i(y_i - f(x_i))^2\right) \] where \(n\) is the number of sampling points, \(w_i\) is the weight, and the data points closer to the mean have higher weights. 2. **Efficient Computation of B - spline Function**: - The paper develops a method for efficiently computing B - spline functions in the HE domain, using techniques such as repeated packing, lazy combination, and comparison functions. For example, through the repeated packing technique, the input vector \(x\) is extended to \([x|x|x\ldots|x]\), and the number of rotation operations is reduced, thus improving the computational efficiency. The specific algorithm is as follows: \[ \text{Algorithm 1: Fast Repeat Packing} \] Input: Ciphertext \(JxK\), grid parameters \(g\) and \(k\), input size \(n_i\) Output: Packed ciphertext \(JxK_{\text{rep}}\) 3. **Experimental Verification**: - The experimental results show that the proposed HE KAN remains competitive in symbolic formula evaluation and image classification tasks. In particular, on the CIFAR - 10 dataset, the inference latency is improved by more than 7 times compared to the naive method. Through these improvements, the paper not only solves the problems of KANs in privacy - protected inference but also significantly improves the inference efficiency and accuracy.

Efficient Privacy-Preserving KAN Inference Using Homomorphic Encryption

Optimized Privacy-Preserving CNN Inference With Fully Homomorphic Encryption

Optimized Layerwise Approximation for Efficient Private Inference on Fully Homomorphic Encryption

Privacy Preserving Inference for Deep Neural Networks: Optimizing Homomorphic Encryption for Efficient and Secure Classification

Efficient and Privacy-Preserving Tree-Based Inference via Additive Homomorphic Encryption

A Secure Convolutional Neural Network Inference Model Based on Homomorphic Encryption

DP-KAN: Differentially Private Kolmogorov-Arnold Networks

Faster CryptoNets: Leveraging Sparsity for Real-World Encrypted Inference

A Privacy-Preserving Computation Framework for Multisource Label Propagation Services

Secure Convolution Neural Network Inference Based on Homomorphic Encryption

Highly Accurate CNN Inference Using Approximate Activation Functions over Homomorphic Encryption

MOFHEI: Model Optimizing Framework for Fast and Efficient Homomorphically Encrypted Neural Network Inference

Batch-oriented Element-wise Approximate Activation for Privacy-Preserving Neural Networks

Accurate Low-Degree Polynomial Approximation of Non-polynomial Operators for Fast Private Inference in Homomorphic Encryption

Hardware Acceleration of Kolmogorov-Arnold Network (KAN) for Lightweight Edge Inference

KAN: Kolmogorov-Arnold Networks

F-KANs: Federated Kolmogorov-Arnold Networks

Self-learning activation functions to increase accuracy of privacy-preserving Convolutional Neural Networks with homomorphic encryption

Efficient Encrypted Inference on Ensembles of Decision Trees

Privacy preserving Neural Network Inference on Encrypted Data with GPUs

Privacy-Preserving Machine Learning With Fully Homomorphic Encryption for Deep Neural Network