Abstract:Event cameras, unlike traditional frame-based cameras, excel in detecting and reporting changes in light intensity on a per-pixel basis. This unique technology offers numerous advantages, including high temporal resolution, low latency, wide dynamic range, and reduced power consumption. These characteristics make event cameras particularly well-suited for sensing applications such as monitoring drivers or human behavior. This paper presents a feasibility study on the using a multitask neural network with event cameras for real-time facial analytics. Our proposed network simultaneously estimates head pose, eye gaze, and facial occlusions. Notably, the network is trained on synthetic event camera data, and its performance is demonstrated and validated using real event data in real-time driving scenarios. To compensate for global head motion, we introduce a novel event integration method capable of handling both short and long-term temporal dependencies. The performance of our facial analytics method is quantitatively evaluated in both controlled lab environments and unconstrained driving scenarios. The results demonstrate that useful accuracy and computational speed is achieved by the proposed method to determining head pose and relative eye-gaze direction. This shows that neuromorphic facial analytics can be realized in real-time and are well-suited for edge/embedded computing deployments. While the improvement ratio in comparison to existing literature may not be as favorable due to the unique event-based vision approach employed, it is crucial to note that our research focuses specifically on event-based vision, which offers distinct advantages over traditional RGB vision. Overall, this study contributes to the emerging field of event-based vision systems and highlights the potential of multitask neural networks combined with event cameras for real-time sensing of human subjects. These techniques can be applied in practical applications such as driver monitoring s- stems, interactive human-computer systems and for human behavior analysis.

NeuroDFD: Towards Efficient Driver Face Detection with Neuromorphic Vision Sensor

ASFD: Automatic and Scalable Face Detector

Driver fatigue detection based on convolutional neural network and face alignment for edge computing device

Real-time detection method of driver fatigue state based on deep learning of face video

An Efficient Face Detector on a CPU Using Dual-Camera Sensors for Intelligent Surveillance Systems

Driver fatigue detection based on comprehensive facial features and gated recurrent unit

R2DS: A novel hierarchical framework for driver fatigue detection in mountain freeway

NeuroIV: Neuromorphic Vision Meets Intelligent Vehicle Towards Safe Driving With a New Database and Baseline Evaluations

Advancing driver fatigue detection in diverse lighting conditions for assisted driving vehicles with enhanced facial recognition technologies

Low-power event-based face detection with asynchronous neuromorphic hardware

Research on a Real-Time Driver Fatigue Detection Algorithm Based on Facial Video Sequences

Real-time face & eye tracking and blink detection using event cameras

Research on Fatigued-Driving Detection Method by Integrating Lightweight YOLOv5s and Facial 3D Keypoints

Real-Time Multi-Task Facial Analytics With Event Cameras

A Fatigue Driving Detection Algorithm Based on Facial Multi-Feature Fusion

Adaptive Driver Face Feature Fatigue Detection Algorithm Research

Driver identification and fatigue detection algorithm based on deep learning

Driver fatigue detection method based on facial features using deep learning

Fatigue Driving Detection Based on Deep Learning and Multi-Index Fusion

MOS: A Low Latency and Lightweight Framework for Face Detection, Landmark Localization, and Head Pose Estimation

Intelligent Driver Warning System using Deep Learning-based Facial Expression Recognition