Abstract:Event cameras, unlike traditional frame-based cameras, excel in detecting and reporting changes in light intensity on a per-pixel basis. This unique technology offers numerous advantages, including high temporal resolution, low latency, wide dynamic range, and reduced power consumption. These characteristics make event cameras particularly well-suited for sensing applications such as monitoring drivers or human behavior. This paper presents a feasibility study on the using a multitask neural network with event cameras for real-time facial analytics. Our proposed network simultaneously estimates head pose, eye gaze, and facial occlusions. Notably, the network is trained on synthetic event camera data, and its performance is demonstrated and validated using real event data in real-time driving scenarios. To compensate for global head motion, we introduce a novel event integration method capable of handling both short and long-term temporal dependencies. The performance of our facial analytics method is quantitatively evaluated in both controlled lab environments and unconstrained driving scenarios. The results demonstrate that useful accuracy and computational speed is achieved by the proposed method to determining head pose and relative eye-gaze direction. This shows that neuromorphic facial analytics can be realized in real-time and are well-suited for edge/embedded computing deployments. While the improvement ratio in comparison to existing literature may not be as favorable due to the unique event-based vision approach employed, it is crucial to note that our research focuses specifically on event-based vision, which offers distinct advantages over traditional RGB vision. Overall, this study contributes to the emerging field of event-based vision systems and highlights the potential of multitask neural networks combined with event cameras for real-time sensing of human subjects. These techniques can be applied in practical applications such as driver monitoring s- stems, interactive human-computer systems and for human behavior analysis.

Towards In-Vehicle Multi-Task Facial Attribute Recognition: Investigating Synthetic Data and Vision Foundation Models

Performance Evaluation of Intelligent Driving Emotion Recognition Model based on Synthetic Dataset in Real Scenes

Face Recognition Using Synthetic Face Data

Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities

Facial Expression Recognition Based on Multi-Scale Convolutional Vision Transformer

SynFace: Face Recognition with Synthetic Data

Real-Time Multi-Task Facial Analytics With Event Cameras

If It's Not Enough, Make It So: Reducing Authentic Data Demand in Face Recognition through Synthetic Faces

Training Deep Face Recognition Systems with Synthetic Data

SDFD: Building a Versatile Synthetic Face Image Dataset with Diverse Attributes

Synthetic Data for Face Recognition: Current State and Future Prospects

Driver Facial Expression Recognition Based on ViT and StarGAN

Driver Multi-task Emotion Recognition Network Based on Multi-modal Facial Video Analysis

Comparative Analysis of Vision Transformer Models for Facial Emotion Recognition Using Augmented Balanced Datasets

Exploring Vision Language Models for Facial Attribute Recognition: Emotion, Race, Gender, and Age

What Do You See in Vehicle? Comprehensive Vision Solution for In-Vehicle Gaze Estimation

Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation

ResAttr-GAN: Unpaired Deep Residual Attributes Learning for Multi-Domain Face Image Translation

SynFER: Towards Boosting Facial Expression Recognition with Synthetic Data

ParallelEye-CS: A New Dataset of Synthetic Images for Testing the Visual Intelligence of Intelligent Vehicles

Attention on Emotions: A Vision Transformer Approach to Advancing Facial Expression Recognition