Kolmogorov-Arnold Networks (KAN) for Time Series Classification and Robust Analysis

Chang Dong,Liangwei Zheng,Weitong Chen

2024-09-11

Abstract:Kolmogorov-Arnold Networks (KAN) has recently attracted significant attention as a promising alternative to traditional Multi-Layer Perceptrons (MLP). Despite their theoretical appeal, KAN require validation on large-scale benchmark datasets. Time series data, which has become increasingly prevalent in recent years, especially univariate time series are naturally suited for validating KAN. Therefore, we conducted a fair comparison among KAN, MLP, and mixed structures. The results indicate that KAN can achieve performance comparable to, or even slightly better than, MLP across 128 time series datasets. We also performed an ablation study on KAN, revealing that the output is primarily determined by the base component instead of b-spline function. Furthermore, we assessed the robustness of these models and found that KAN and the hybrid structure MLP\_KAN exhibit significant robustness advantages, attributed to their lower Lipschitz constants. This suggests that KAN and KAN layers hold strong potential to be robust models or to improve the adversarial robustness of other models.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

The main objective of this paper is to validate the performance of Kolmogorov-Arnold Networks (KAN) in time series classification tasks and to assess their robustness. Specifically:

1. **Performance Comparison**: The paper explores the performance differences between KAN and traditional Multi-Layer Perceptrons (MLP) as well as hybrid structures through a fair comparison on 128 UCR datasets. The results indicate that KAN can achieve performance comparable to or slightly better than MLP on these datasets.

2. **Component Analysis**: The study also conducted ablation experiments to analyze the roles of different components in KAN, particularly the base function and the B-spline function. The results show that the base function has a more significant impact on the output, while larger grid sizes may lead to optimization difficulties.

3. **Robustness Evaluation**: The paper further evaluates the robustness of KAN and other models under adversarial attacks. The study finds that KAN exhibits significant adversarial robustness, which is attributed to its lower Lipschitz constant. Interestingly, larger grid sizes, although leading to higher Lipschitz constants, demonstrate stronger robustness.

In summary, the paper aims to validate the effectiveness and robustness of KAN as a new neural network architecture in time series classification tasks, and to explore its internal mechanisms and performance in adversarial environments.