Abstract:The recently proposed Kolmogorov-Arnold Networks (KAN) networks have attracted increasing attention due to their advantage of high visualizability compared to MLP. In this paper, based on a series of small-scale experiments, we proposed the Efficient KAN Expansion Principle (EKE Principle): allocating parameters to expand network scale, rather than employing more complex basis functions, leads to more efficient performance improvements in KANs. Based on this principle, we proposed a superior KAN termed SKAN, where the basis function utilizes only a single learnable parameter. We then evaluated various single-parameterized functions for constructing SKANs, with LShifted Softplus-based SKANs (LSS-SKANs) demonstrating superior accuracy. Subsequently, extensive experiments were performed, comparing LSS-SKAN with other KAN variants on the MNIST dataset. In the final accuracy tests, LSS-SKAN exhibited superior performance on the MNIST dataset compared to all tested pure KAN variants. Regarding execution speed, LSS-SKAN outperformed all compared popular KAN variants. Our experimental codes are available at <a class="link-external link-https" href="https://github.com/chikkkit/LSS-SKAN" rel="external noopener nofollow">this https URL</a> and SKAN's Python library (for quick construction of SKAN in python) codes are available at <a class="link-external link-https" href="https://github.com/chikkkit/SKAN" rel="external noopener nofollow">this https URL</a> .

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to improve the performance and efficiency of Kolmogorov - Arnold Networks (KAN). Specifically, the author proposes a new principle named Efficient KAN Expansion Principle (EKE Principle), that is, by allocating parameters to expand the network scale instead of using more complex basis functions, the performance of KAN can be improved more effectively. Based on this principle, the author designs a new KAN variant called Single - Parameterized KAN (SKAN), in which the basis function contains only one learnable parameter. Further, the author evaluates a variety of single - parameterized functions and finds that SKAN based on the Learnable Shifted Softplus (LSS) function (LSS - SKAN) performs excellently. To verify the effectiveness of LSS - SKAN, the author conducts extensive experiments on the MNIST dataset and compares it with existing KAN variants (such as Wav - KAN, Spl - KAN, FastKAN, MLP + rKAN, MLP + fKAN, etc.). The experimental results show that LSS - SKAN is superior to other pure KAN variants in terms of accuracy and execution speed. ### Main Contributions 1. **Propose the EKE principle**: By allocating parameters to expand the network scale instead of using more complex basis functions, the performance of KAN can be improved more effectively. 2. **Design SKAN**: Propose a new KAN variant whose basis function contains only one learnable parameter. 3. **Select the best basis function**: Through experimental evaluation, it is found that the LSS function is the optimal single - parameterized function. 4. **Experimental proof**: The experimental results on the MNIST dataset show that LSS - SKAN is superior to other KAN variants in terms of accuracy and execution speed. ### Experimental Setup - **Task**: MNIST handwritten digit recognition - **Dataset**: MNIST dataset - **Optimizer**: Adam - **Learning rate search range**: [0.001, 0.009], [0.01, 0.09], [0.1, 1] - **Network structure**: A two - layer network, with an input size of 784 (corresponding to a 28x28 MNIST image) and an output size of 10 (representing 10 handwritten digits) - **Hardware configuration**: The CPU is "Intel(R) Core(TM) i7 - 6850K @ 3.60GHz", the GPU is "GeForce RTX 3090", and the operating system is Ubuntu 22.04.1 ### Experimental Results - **Performance comparison**: The accuracy and F1 - score of LSS - SKAN on the test set are both better than those of other KAN variants. - **Training time**: The training time of LSS - SKAN is also shorter than that of other KAN variants. In conclusion, this paper significantly improves the performance and efficiency of KAN by proposing the EKE principle and designing LSS - SKAN, providing a new direction for the development of KAN.

LSS-SKAN: Efficient Kolmogorov-Arnold Networks based on Single-Parameterized Function

LArctan-SKAN: Simple and Efficient Single-Parameterized Kolmogorov-Arnold Networks using Learnable Trigonometric Function

KAN: Kolmogorov-Arnold Networks

Activation Space Selectable Kolmogorov-Arnold Networks

On the expressiveness and spectral bias of KANs

KKANs: Kurkova-Kolmogorov-Arnold Networks and Their Learning Dynamics

EKAN: Equivariant Kolmogorov-Arnold Networks

BSRBF-KAN: A combination of B-splines and Radial Basis Functions in Kolmogorov-Arnold Networks

Can KAN Work? Exploring the Potential of Kolmogorov-Arnold Networks in Computer Vision

A Revolutionary Neural Network Architecture with Interpretability and Flexibility Based on Kolmogorov–Arnold for Solar Radiation and Temperature Forecasting

KANs for Computer Vision: An Experimental Study

FC-KAN: Function Combinations in Kolmogorov-Arnold Networks

Kolmogorov-Arnold Networks (KAN) for Time Series Classification and Robust Analysis

PowerMLP: An Efficient Version of KAN

Kolmogorov-Arnold Networks for Genomic Tasks

How to Learn More? Exploring Kolmogorov-Arnold Networks for Hyperspectral Image Classification

Incorporating Arbitrary Matrix Group Equivariance into KANs

DKL-KAN: Scalable Deep Kernel Learning using Kolmogorov-Arnold Networks

Demonstrating the Efficacy of Kolmogorov-Arnold Networks in Vision Tasks

Kolmogorov-Arnold Convolutions: Design Principles and Empirical Studies

Efficient prediction of potential energy surface and physical properties with Kolmogorov-Arnold Networks