P1-KAN an effective Kolmogorov Arnold Network for function approximation

Xavier Warin

2024-10-23

Abstract:A new Kolmogorov-Arnold network (KAN) is proposed to approximate potentially irregular functions in high dimension. We show that it outperforms multilayer perceptrons in terms of accuracy and converges faster. We also compare it with several proposed KAN networks: the original spline-based KAN network appears to be more effective for smooth functions, while the P1-KAN network is more effective for irregular functions.

Machine Learning,Neural and Evolutionary Computing

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to effectively approximate potentially irregular functions in high - dimensional spaces. Specifically, the author proposes a new Kolmogorov - Arnold network (P1 - KAN), aiming to overcome the limitations of existing methods in dealing with irregular functions and improve the accuracy and convergence speed of function approximation.

### Background
The Kolmogorov - Arnold representation theorem states that a multivariate continuous and smooth function on a bounded domain can be represented as a finite combination of univariate continuous functions. The specific form is as follows:
\[ f(\mathbf{x})=\sum_{i = 1}^{2n+1}\psi_i\left(\sum_{j = 1}^n\Phi_{i,j}(x_j)\right) \]
where \(\Phi_{i,j}:[0,1]\to\mathbb{R}\) and \(\psi_i:\mathbb{R}\to\mathbb{R}\).

### Limitations of Existing Methods
1. **Spline Approximation**: Although spline approximation is theoretically effective, in practical applications, the computational cost is high, and the output may not be within the initially selected grid, requiring dynamic adjustment of the grid, which increases complexity and computational cost.
2. **Irregular Functions**: The Kolmogorov - Arnold representation theorem involves very irregular functions, and using high - order approximations (such as splines) may not be the best choice.

### Characteristics of the P1 - KAN Network
1. **P1 Finite Element Method**: The P1 - KAN network uses the P1 finite element method to define the layer functions \(\Phi_i\), and approximates the function through the linear interpolation basis functions \(\Psi_i\).
2. **Explicit Definition of Support Domain**: The P1 - KAN network not only inputs samples \(\mathbf{x}\), but also inputs the description \((x_i,\bar{x}_i)\) of the support domain.
3. **Avoid Network Adaptation**: The P1 - KAN network avoids dynamic adjustment of the grid, simplifying the computational process.

### Numerical Results
The paper verifies the effectiveness of the P1 - KAN network by comparing the performance of the P1 - KAN network with that of the multi - layer perceptron (MLP), spline KAN, fast KAN and ReLU - KAN networks in different dimensions. The main results are as follows:
- **Smooth Functions**: For smooth but rapidly oscillating functions, the spline KAN network performs best, followed by the P1 - KAN network.
- **Irregular Functions**: For very irregular functions, the P1 - KAN network performs best, especially in high - dimensional spaces.

### Conclusion
The P1 - KAN network performs excellently in approximating irregular functions, especially in high - dimensional spaces. Although it is not as good as the spline KAN network in approximating smooth functions, it has obvious advantages in dealing with irregular functions. The paper also points out that the P1 - KAN network can effectively avoid the divergence problems encountered by the ReLU - KAN network in practical applications.