Abstract:This paper introduces a new system for dynamic visual recognition that combines bio-inspired hardware with a brain-like spiking neural network. The system is designed to take data from a dynamic vision sensor (DVS) that simulates the functioning of the human retina by producing an address event output (spike trains) based on the movement of objects. The system then convolutes the spike trains and feeds them into a brain-like spiking neural network, called NeuCube, which is organized in a three-dimensional manner, representing the organization of the primary visual cortex. Spatio-temporal patterns of the data are learned during a deep unsupervised learning stage, using spike-timing-dependent plasticity. In a second stage, supervised learning is performed to train the network for classification tasks. The convolution algorithm and the mapping into the network mimic the function of retinal ganglion cells and the retinotopic organization of the visual cortex. The NeuCube architecture can be used to visualize the deep connectivity inside the network before, during, and after training and thereby allows for a better understanding of the learning processes. The method was tested on the benchmark MNIST-DVS dataset and achieved a classification accuracy of 92.90%. The paper discusses advantages and limitations of the new method and concludes that it is worth exploring further on different datasets, aiming for advances in dynamic computer vision and multimodal systems that integrate visual, aural, tactile, and other kinds of information in a biologically plausible way.

Visual Pattern Recognition with on On-Chip Learning: Towards a Fully Neuromorphic Approach

Neuromorphic Hardware System for Visual Pattern Recognition With Memristor Array and CMOS Neuron

Pattern Recognition Computation in a Spiking Neural Network with Temporal Encoding and Learning

A Compact and Accelerated Spike-based Neuromorphic VLSI Chip for Pattern Recognition.

In-Hardware Learning of Multilayer Spiking Neural Networks on a Neuromorphic Processor

An error-propagation spiking neural network compatible with neuromorphic processors

A Neurally Inspired Pattern Recognition Approach with Latency-Phase Encoding and Precise-Spike-driven Rule in Spiking Neural Network.

Rapid Feedforward Computation by Temporal Encoding and Learning with Spiking Neurons

TripleBrain: An Edge Neuromorphic Architecture for High-accuracy Single-layer Spiking Neural Network with On-chip Self-organizing and Reinforcement Learning

A Spiking Network for Inference of Relations Trained with Neuromorphic Backpropagation

Homogeneous Spiking Neuromorphic System for Real-World Pattern Recognition

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

Spiking-timing based pattern recognition with real-world visual stimuli

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

A Multichip Neuromorphic System for Spike-Based Visual Information Processing

Neuromorphic analog circuits for robust on-chip always-on learning in spiking neural networks

Scalable NoC-based Neuromorphic Hardware Learning and Inference

A Retinotopic Spiking Neural Network System for Accurate Recognition of Moving Objects Using NeuCube and Dynamic Vision Sensors

Unsupervised Learning of Spike-Timing-Dependent Plasticity Based on a Neuromorphic Implementation

An Efficient Neuromorphic Implementation of Temporal Coding-Based On-Chip STDP Learning