Abstract:With the success of deep learning, object recognition systems that can be deployed for real-world applications are becoming commonplace. However, inference that needs to largely take place on the `edge' (not processed on servers), is a highly computational and memory intensive workload, making it intractable for low-power mobile nodes and remote security applications. To address this challenge, this paper proposes a low-power (5W) end-to-end neuromorphic framework for object tracking and classification using event-based cameras that possess desirable properties such as low power consumption (5-14 mW) and high dynamic range (120 dB). Nonetheless, unlike traditional approaches of using event-by-event processing, this work uses a mixed frame and event approach to get energy savings with high performance. Using a frame-based region proposal method based on the density of foreground events, a hardware-friendly object tracking is implemented using the apparent object velocity while tackling occlusion scenarios. For low-power classification of the tracked objects, the event camera is interfaced to IBM TrueNorth, which is time-multiplexed to tackle up to eight instances for a traffic monitoring application. The frame-based object track input is converted back to spikes for Truenorth classification via the energy efficient deep network (EEDN) pipeline. Using originally collected datasets, we train the TrueNorth model on the hardware track outputs, instead of using ground truth object locations as commonly done, and demonstrate the efficacy of our system to handle practical surveillance scenarios. Finally, we compare the proposed methodologies to state-of-the-art event-based systems for object tracking and classification, and demonstrate the use case of our neuromorphic approach for low-power applications without sacrificing on performance.

Low-power multi-camera object re-identification using hierarchical neural networks

Deep Siamese Network with Multi-level Similarity Perception for Person Re-identification

Hierarchical and Efficient Learning for Person Re-Identification

Person Re-identification Based on Transform Algorithm

Survey on Energy-Efficient Deep Neural Networks for Computer Vision

A low-power end-to-end hybrid neuromorphic framework for surveillance applications

Multi-level Similarity Perception Network for Person Re-identification

Multi-camera Handoff for Person Re-Identification

Person re-identification on lightweight devices: end-to-end approach

Object Re-Identification from Point Clouds

Learning to Align Multi-Camera Domains using Part-Aware Clustering for Unsupervised Video Person Re-Identification

PersonNet: Person Re-identification with Deep Convolutional Neural Networks

Multi-scale Deep Learning Architectures for Person Re-identification

HyNNA: Improved Performance for Neuromorphic Vision Sensor based Surveillance using Hybrid Neural Network Architecture

Dual Branch Attention Network for Person Re-Identification

Multi-Level Fusion Temporal-Spatial Co-Attention for Video-Based Person Re-Identification

Auto-ReID: Searching for a Part-Aware ConvNet for Person Re-Identification

Deep Low-Resolution Person Re-Identification

Information complementary attention-based multidimension feature learning for person re-identification

Consistent-Aware Deep Learning For Person Re-Identification In A Camera Network

Multi-Task Learning with Low Rank Attribute Embedding for Multi-Camera Person Re-Identification