Abstract:Event cameras, unlike traditional frame-based cameras, excel in detecting and reporting changes in light intensity on a per-pixel basis. This unique technology offers numerous advantages, including high temporal resolution, low latency, wide dynamic range, and reduced power consumption. These characteristics make event cameras particularly well-suited for sensing applications such as monitoring drivers or human behavior. This paper presents a feasibility study on the using a multitask neural network with event cameras for real-time facial analytics. Our proposed network simultaneously estimates head pose, eye gaze, and facial occlusions. Notably, the network is trained on synthetic event camera data, and its performance is demonstrated and validated using real event data in real-time driving scenarios. To compensate for global head motion, we introduce a novel event integration method capable of handling both short and long-term temporal dependencies. The performance of our facial analytics method is quantitatively evaluated in both controlled lab environments and unconstrained driving scenarios. The results demonstrate that useful accuracy and computational speed is achieved by the proposed method to determining head pose and relative eye-gaze direction. This shows that neuromorphic facial analytics can be realized in real-time and are well-suited for edge/embedded computing deployments. While the improvement ratio in comparison to existing literature may not be as favorable due to the unique event-based vision approach employed, it is crucial to note that our research focuses specifically on event-based vision, which offers distinct advantages over traditional RGB vision. Overall, this study contributes to the emerging field of event-based vision systems and highlights the potential of multitask neural networks combined with event cameras for real-time sensing of human subjects. These techniques can be applied in practical applications such as driver monitoring s- stems, interactive human-computer systems and for human behavior analysis.

Real-time crowd behavior recognition in surveillance videos based on deep learning methods

Crowd dynamics analysis and behavior recognition in surveillance videos based on deep learning

Real-Time Target Detection and Recognition with Deep Convolutional Networks for Intelligent Visual Surveillance

Unsupervised group-based crowd dynamic behavior detection and tracking in online video sequences

Human Action Recognition From Digital Videos Based on Deep Learning.

Human crowd behaviour analysis based on video segmentation and classification using expectation–maximization with deep learning architectures

Abnormal Activity Recognition Based on Deep Learning in Crowd

Crowd Characterization in Surveillance Videos Using Deep-Graph Convolutional Neural Network

Human Action Recognition Using Deep Learning Methods.

An Abnormal Behavior Detection Method of Video Crowds and Vehicles Based on Deep Learning

Exploring Human Crowd Patterns and Categorization in Video Footage for Enhanced Security and Surveillance using Computer Vision and Machine Learning

Security Event Recognition for Visual Surveillance

PublicVision: A Secure Smart Surveillance System for Crowd Behavior Recognition

Public Place Crowd Transaction Monitoring System

Plug-and-Play CNN for Crowd Motion Analysis: An Application in Abnormal Event Detection

Abnormal Event Detection in Urban Surveillance Videos Using GAN and Transfer Learning

Real-Time Multi-Task Facial Analytics With Event Cameras

A novel multi-scale violence and public gathering dataset for crowd behavior classification

Hybrid Classifiers for Spatio-temporal Real-time Abnormal Behaviors Detection, Tracking, and Recognition in Massive Hajj Crowds

Deep Learning-Based Real-Time Multiple-Person Action Recognition System

Zero-Shot Crowd Behavior Recognition