Kolmogorov-Arnold Network for Online Reinforcement Learning

Victor Augusto Kich,Jair Augusto Bottega,Raul Steinmetz,Ricardo Bedin Grando,Ayano Yorozu,Akihisa Ohya
2024-09-01
Abstract:Kolmogorov-Arnold Networks (KANs) have shown potential as an alternative to Multi-Layer Perceptrons (MLPs) in neural networks, providing universal function approximation with fewer parameters and reduced memory usage. In this paper, we explore the use of KANs as function approximators within the Proximal Policy Optimization (PPO) algorithm. We evaluate this approach by comparing its performance to the original MLP-based PPO using the DeepMind Control Proprio Robotics benchmark. Our results indicate that the KAN-based reinforcement learning algorithm can achieve comparable performance to its MLP-based counterpart, often with fewer parameters. These findings suggest that KANs may offer a more efficient option for reinforcement learning models.
Machine Learning,Artificial Intelligence
What problem does this paper attempt to address?
The paper aims to explore the potential application of Kolmogorov-Arnold Networks (KANs) in reinforcement learning, particularly as function approximators used in the Proximal Policy Optimization (PPO) algorithm. Specifically, the researchers hope to verify whether KANs can reduce the number of parameters while maintaining or improving performance by comparing them with traditional Multi-Layer Perceptrons (MLPs), thereby enhancing model efficiency. The experiments were evaluated on multiple robotic control tasks, and the results indicate that KANs can indeed achieve comparable or even better performance than MLPs in certain tasks, with a significant reduction in the number of parameters. This suggests that KANs may become a more efficient model choice for reinforcement learning. However, despite the parameter efficiency advantage of KANs, there are still shortcomings in computational speed, necessitating further research to optimize their computational efficiency.