Abstract:We present the first purely event-based, energy-efficient approach for dynamic object detection and categorization with a freely moving event camera. Compared to traditional cameras, event-based object recognition systems are considerably behind in terms of accuracy and algorithmic maturity. To this end, this paper presents an event-based feature extraction method devised by accumulating local activity across the image frame and then applying principal component analysis (PCA) to the normalized neighborhood region. Subsequently, we propose a backtracking-free k-d tree mechanism for efficient feature matching by taking advantage of the low-dimensionality of the feature representation. Additionally, the proposed k-d tree mechanism allows for feature selection to obtain a lower-dimensional object representation when hardware resources are limited to implement PCA. Consequently, the proposed system can be realized on a field-programmable gate array (FPGA) device leading to high performance over resource ratio. The proposed system is tested on real-world event-based datasets for object categorization, showing superior classification performance compared to state-of-the-art algorithms. Additionally, we verified the real-time FPGA performance of the proposed object detection method, trained with limited data as opposed to deep learning methods, under a closed-loop aerial vehicle flight mode. We also compare the proposed object categorization framework to pre-trained convolutional neural networks using transfer learning and highlight the drawbacks of using frame-based sensors under dynamic camera motion. Finally, we provide critical insights about the feature extraction method and the classification parameters on the system performance, which aids in understanding the framework to suit various low-power (less than a few watts) application scenarios.

A bio-inspired feedforward system for categorization of AER motion events

Feedforward Categorization on AER Motion Events Using Cortex-Like Features in a Spiking Neural Network

Bio-inspired Categorization Using Event-Driven Feature Extraction and Spike-Based Learning.

An Event-based Categorization Model Using Spatio-temporal Features in a Spiking Neural Network.

An Event-Driven Categorization Model for AER Image Sensors Using Multispike Encoding and Learning

An Event-Driven Object Recognition Model Using Activated Connected Domain Detection

An Event-Driven Computational System With Spiking Neurons For Object Recognition

Bag of Events: an Efficient Probability-Based Feature Extraction Method for AER Image Sensors

An Event-Based Neurobiological Recognition System with Orientation Detector for Objects in Multiple Orientations

Effective AER Object Classification Using Segmented Probability-Maximization Learning in Spiking Neural Networks

Recognition for AER Objects Based on Hierarchical Feature Construction

Efficient Motion Symbol Detection and Multi-kernel Learning for AER Object Recognition

Neural Implementation of Categorization in a Motion Discrimination Task

EventF2S: Asynchronous and Sparse Spiking AER Framework using Neuromorphic-Friendly Algorithm

A Bio-Inspired Hierarchical Spiking Neural Network With Reward-Modulated STDP Learning Rule for AER Object Recognition

An Event-based Feature Representation Method for Event Stream Classification Using Deep Spiking Neural Networks

High-speed Classification of AER Data Based on a Low-cost Hardware System.

A Neuro-Cognitive System and Its Application in Robotics

Event-based Action Recognition Using Motion Information and Spiking Neural Networks

Unsupervised AER Object Recognition Based on Multiscale Spatio-Temporal Features and Spiking Neurons

Low-Power Dynamic Object Detection and Classification With Freely Moving Event Cameras