Abstract:Artificial neural networks (ANNs), a popular path towards artificial intelligence, have experienced remarkable success via mature models, various benchmarks, open-source datasets, and powerful computing platforms. Spiking neural networks (SNNs), a category of promising models to mimic the neuronal dynamics of the brain, have gained much attention for brain inspired computing and been widely deployed on neuromorphic devices. However, for a long time, there are ongoing debates and skepticisms about the value of SNNs in practical applications. Except for the low power attribute benefit from the spike-driven processing, SNNs usually perform worse than ANNs especially in terms of the application accuracy. Recently, researchers attempt to address this issue by borrowing learning methodologies from ANNs, such as backpropagation, to train high-accuracy SNN models. The rapid progress in this domain continuously produces amazing results with ever-increasing network size, whose growing path seems similar to the development of deep learning. Although these ways endow SNNs the capability to approach the accuracy of ANNs, the natural superiorities of SNNs and the way to outperform ANNs are potentially lost due to the use of ANN-oriented workloads and simplistic evaluation metrics. In this paper, we take the visual recognition task as a case study to answer the questions of "what workloads are ideal for SNNs and how to evaluate SNNs makes sense". We design a series of contrast tests using different types of datasets (ANN-oriented and SNN-oriented), diverse processing models, signal conversion methods, and learning algorithms. We propose comprehensive metrics on the application accuracy and the cost of memory & compute to evaluate these models, and conduct extensive experiments. We evidence the fact that on ANN-oriented workloads, SNNs fail to beat their ANN counterparts; while on SNN-oriented workloads, SNNs can fully perform better. We further demonstrate that in SNNs there exists a trade-off between the application accuracy and the execution cost, which will be affected by the simulation time window and firing threshold. Based on these abundant analyses, we recommend the most suitable model for each scenario. To the best of our knowledge, this is the first work using systematical comparisons to explicitly reveal that the straightforward workload porting from ANNs to SNNs is unwise although many works are doing so and a comprehensive evaluation indeed matters. Finally, we highlight the urgent need to build a benchmarking framework for SNNs with broader tasks, datasets, and metrics.

Performance Analysis and Comparison of Neural Networks and Support Vector Machines Classifier

An Unbalanced Dataset Classification Approach Based On V-Support Vector Machine

A PERFORMANCE PREDICTION METHOD BASED ON NAIVE BAYES CLASSIFICATION

SVM Classification:Its Contents and Challenges

Comparison of Support Vector Machine and Softmax Classifiers in Computer Vision

Basic Tenets of Classification Algorithms K-Nearest-Neighbor, Support Vector Machine, Random Forest and Neural Network: A Review

Examining the Use of Neural Networks for Feature Extraction: A Comparative Analysis using Deep Learning, Support Vector Machines, and K-Nearest Neighbor Classifiers

Comparative analysis of image classification algorithms based on traditional machine learning and deep learning

Research and Analysis of Methods for Multiclass Support Vector Machines

Comparative Analysis For Probability Modeling Of Multi-Class Svm

Rethinking the Performance Comparison Between SNNS and ANNS.

A Comparative Analysis of Support Vector Machines and Extreme Learning Machines

Comparative study on classification performance between support vector machine and logistic regression

Mining Knowledge from Unbalanced Data: Effect of Class Distribution on SVM Classification

Research on the Assessment for Air Environment Quality Based on Support Vector Machine

Research of Support Vector Machines

Comparing two SVM models through different metrics based on the confusion matrix

Performance Comparison of Supervised Learning Using Non-Neural Network and Neural Network

Research on the improvement effect of machine learning and neural network algorithms on the prediction of learning achievement

Performance Analysis of Deep Learning and Machine Learning Methods for Music Genre Classification System

Comparison of Support Vector Machine and Back Propagation Neural Network in Evaluating the Enterprise Financial Distress