Abstract:Neuromorphic vision sensor is a new bio-inspired imaging paradigm that reports asynchronous, continuously per-pixel brightness changes called `events' with high temporal resolution and high dynamic range. So far, the event-based image reconstruction methods are based on artificial neural networks (ANN) or hand-crafted spatiotemporal smoothing techniques. In this paper, we first implement the image reconstruction work via fully spiking neural network (SNN) architecture. As the bio-inspired neural networks, SNNs operating with asynchronous binary spikes distributed over time, can potentially lead to greater computational efficiency on event-driven hardware. We propose a novel Event-based Video reconstruction framework based on a fully Spiking Neural Network (EVSNN), which utilizes Leaky-Integrate-and-Fire (LIF) neuron and Membrane Potential (MP) neuron. We find that the spiking neurons have the potential to store useful temporal information (memory) to complete such time-dependent tasks. Furthermore, to better utilize the temporal information, we propose a hybrid potential-assisted framework (PA-EVSNN) using the membrane potential of spiking neuron. The proposed neuron is referred as Adaptive Membrane Potential (AMP) neuron, which adaptively updates the membrane potential according to the input spikes. The experimental results demonstrate that our models achieve comparable performance to ANN-based models on IJRR, MVSEC, and HQF datasets. The energy consumptions of EVSNN and PA-EVSNN are 19.36$\times$ and 7.75$\times$ more computationally efficient than their ANN architectures, respectively.

DVS-Voltmeter: Stochastic Process-Based Event Simulator for Dynamic Vision Sensors.

V2CE: Video to Continuous Events Simulator

Enhanced Frame and Event-Based Simulator and Event-Based Video Interpolation Network

Denoising Method for Dynamic Vision Sensor Based on Two-Dimensional Event Density

A dynamic vision sensor object recognition model based on trainable event-driven convolution and spiking attention mechanism

An Event Coding Method Based on Frame Images with Dynamic Vision Sensor Modeling

v2e: From Video Frames to Realistic DVS Events

Exploiting Alternating DVS Shot Noise Event Pair Statistics to Reduce Background Activity

Supervised Nonlinear Dynamic System for Soft Sensor Application Aided by Variational Auto-Encoder

A 1024-Neuron 1M-Synapse Event-Driven SNN Accelerator for DVS Applications

Deep Learning of Process Data with Supervised Variational Auto-encoder for Soft Sensor

An SNN-Based and Neuromorphic-Hardware-Implementable Noise Filter with Self-adaptive Time Window for Event-Based Vision Sensor.

DVS-Attacks: Adversarial Attacks on Dynamic Vision Sensors for Spiking Neural Networks

Event-based Video Reconstruction via Potential-assisted Spiking Neural Network

Deep Representation Via Convolutional Neural Network for Classification of Spatiotemporal Event Streams

Unsupervised Learning of Dense Optical Flow, Depth and Egomotion from Sparse Event Data

Characterization of dim light response in DVS pixel: Discontinuity of event triggering time

A Noise Filter for Dynamic Vision Sensors using Self-adjusting Threshold

EventZoom: A Progressive Approach to Event-Based Data Augmentation for Enhanced Neuromorphic Vision

Adaptive Slicing Method of the Spatiotemporal Event Stream Obtained from a Dynamic Vision Sensor

Temporal-Mapping Photography for Event Cameras