Abstract:<p>Neuromorphic data, recording frameless spike events, have attracted considerable attention for the spatiotemporal information components and the event-driven processing fashion. Spiking neural networks (SNNs) represent a family of event-driven models with spatiotemporal dynamics for neuromorphic computing, which are widely benchmarked on neuromorphic data. Interestingly, researchers in the machine learning community can argue that recurrent (artificial) neural networks (RNNs) also have the capability to extract spatiotemporal features although they are not event-driven. Thus, the question of "what will happen if we benchmark these two kinds of models together on neuromorphic data" comes out but remains unclear.</p><p>In this work, we make a systematic study to compare SNNs and RNNs on neuromorphic data, taking the vision datasets as a case study. First, we identify the similarities and differences between SNNs and RNNs (including the vanilla RNNs and LSTM) from the modeling and learning perspectives. To improve comparability and fairness, we unify the supervised learning algorithm based on backpropagation through time (BPTT), the loss function exploiting the outputs at all timesteps, the network structure with stacked fully-connected or convolutional layers, and the hyper-parameters during training. Especially, given the mainstream loss function used in RNNs, we modify it inspired by the rate coding scheme to approach that of SNNs. Furthermore, we tune the temporal resolution of datasets to test model robustness and generalization. At last, a series of contrast experiments are conducted on two types of neuromorphic datasets: DVS-converted (N-MNIST) and DVS-captured (DVS Gesture). Extensive insights regarding recognition accuracy, feature extraction, temporal resolution and contrast, learning generalization, computational complexity and parameter volume are provided, which are beneficial for the model selection on different workloads and even for the invention of novel neural models in the future.</p>

Survey of Current Progress in Random Neural Network

Application study of a kind of novel neural networks models

RNN algorithm optimization based on extended unsaturated region

Improved algorithm research of optimization solution with random neural network

Advances in Convolutional Neural Networks

Comparison Study of Optimization Solution between Random Neural Network and Hopfield Network

Deep Learning with the Random Neural Network and its Applications

Application of the Edge of Chaos in Combinatorial Optimization*

Solving C-TSP with random neural network

A fast algorithm for solving large scale nonlinear optimization problems using RNN

Study on RAN Network and it's Application

Random vector functional link network: Recent developments, applications, and future directions

A Survey of Randomized Algorithms for Training Neural Networks.

A Review of Recent Advances and Application for Spiking Neural Networks

Robust noise-aware algorithm for randomized neural network and its convergence properties

Optimization and Application of Communication Resource Allocation Algorithm for Urban Rail Transit Planning

Comparing SNNs and RNNs on neuromorphic vision datasets: Similarities and differences

Random-Coupled Neural Network

Survey on the attention based RNN model and its applications in computer vision

Dynamic Neural Networks: A Survey