Abstract:In the realm of deep learning, the Kolmogorov-Arnold Network (KAN) has emerged as a potential alternative to multilayer projections (MLPs). However, its applicability to vision tasks has not been extensively validated. In our study, we demonstrated the effectiveness of KAN for vision tasks through multiple trials on the MNIST, CIFAR10, and CIFAR100 datasets, using a training batch size of 32. Our results showed that while KAN outperformed the original MLP-Mixer on CIFAR10 and CIFAR100, it performed slightly worse than the state-of-the-art ResNet-18. These findings suggest that KAN holds significant promise for vision tasks, and further modifications could enhance its performance in future evaluations.Our contributions are threefold: first, we showcase the efficiency of KAN-based algorithms for visual tasks; second, we provide extensive empirical assessments across various vision benchmarks, comparing KAN's performance with MLP-Mixer, CNNs, and Vision Transformers (ViT); and third, we pioneer the use of natural KAN layers in visual tasks, addressing a gap in previous research. This paper lays the foundation for future studies on KANs, highlighting their potential as a reliable alternative for image classification tasks.

What problem does this paper attempt to address?

The problem this paper attempts to address is the validation of the effectiveness of the Kolmogorov-Arnold Network (KAN) in visual tasks. Specifically, the researchers aim to demonstrate the performance of KAN in image classification tasks by using the KAN-Mixer architecture on multiple datasets (such as MNIST, CIFAR10, and CIFAR100) and comparing it with existing models like Multilayer Perceptron (MLP), Convolutional Neural Networks (CNN), and Vision Transformers (ViT). ### Main Issues: 1. **Applicability of KAN in Visual Tasks**: Although KAN has shown potential in other fields, its application in visual tasks has not been fully validated. 2. **Performance Comparison of KAN with Existing Models**: Evaluate the performance of KAN-Mixer relative to MLP-Mixer, CNN, and ViT models through experiments on multiple benchmark datasets. 3. **Potential Advantages of KAN**: Explore the possible advantages of KAN in visual tasks, especially its performance on complex datasets. ### Research Methods: - **Datasets**: Experiments are conducted using the MNIST, CIFAR10, and CIFAR100 datasets. - **Model Architecture**: The KAN-Mixer architecture is adopted, which uses only KAN layers without additional algorithms like CNN or ViT. - **Experimental Setup**: A training batch size of 32 is used, hyperparameters (such as `n_channels` and `n_hiddens`) are systematically adjusted, and the model's performance is evaluated on multiple datasets. ### Experimental Results: - **MNIST**: KAN-Mixer achieved a test accuracy of 98.16% on the MNIST dataset, outperforming MLP-Mixer and ViT-10/4 but slightly lower than ResNet-18. - **CIFAR10**: On the CIFAR10 dataset, KAN-Mixer's test accuracy was 66.93%, better than MLP-Mixer but significantly lower than ResNet-18. - **CIFAR100**: On the CIFAR100 dataset, KAN-Mixer's test accuracy was 35.49%, also better than MLP-Mixer but lower than ResNet-18 and ViT-10/4. ### Conclusion: - **Potential of KAN**: The study indicates that KAN has significant potential in visual tasks, performing well on simpler datasets like MNIST. - **Future Improvement Directions**: Although KAN's performance on complex datasets is not as good as the state-of-the-art CNN models, further tuning and improvements could lead to better performance in future visual tasks. Overall, this paper aims to experimentally demonstrate the effectiveness and potential of KAN in visual tasks, laying the groundwork for further research.

Demonstrating the Efficacy of Kolmogorov-Arnold Networks in Vision Tasks

Can KAN Work? Exploring the Potential of Kolmogorov-Arnold Networks in Computer Vision

A preliminary study on continual learning in computer vision using Kolmogorov-Arnold Networks

Kolmogorov-Arnold Network Autoencoders

KANs for Computer Vision: An Experimental Study

Convolutional Kolmogorov-Arnold Networks

Kolmogorov-Arnold Convolutions: Design Principles and Empirical Studies

Residual Kolmogorov-Arnold Network for Enhanced Deep Learning

Kolmogorov-Arnold Network for Satellite Image Classification in Remote Sensing

KAN: Kolmogorov-Arnold Networks

A Benchmarking Study of Kolmogorov-Arnold Networks on Tabular Data

KANICE: Kolmogorov-Arnold Networks with Interactive Convolutional Elements

Suitability of KANs for Computer Vision: A preliminary investigation

How to Learn More? Exploring Kolmogorov–Arnold Networks for Hyperspectral Image Classification

HyperKAN: Kolmogorov-Arnold Networks make Hyperspectral Image Classificators Smarter

A Revolutionary Neural Network Architecture with Interpretability and Flexibility Based on Kolmogorov–Arnold for Solar Radiation and Temperature Forecasting

How to Learn More? Exploring Kolmogorov-Arnold Networks for Hyperspectral Image Classification

On Training of Kolmogorov-Arnold Networks

On the expressiveness and spectral bias of KANs

Kolmogorov-Arnold Networks (KAN) for Time Series Classification and Robust Analysis

Exploring the Limitations of Kolmogorov-Arnold Networks in Classification: Insights to Software Training and Hardware Implementation