Abstract:To further advance driver monitoring and assistance systems, it is important to understand how drivers allocate their attention, in other words, where do they tend to look and why. Traditionally, factors affecting human visual attention have been divided into bottom-up (involuntary attraction to salient regions) and top-down (driven by the demands of the task being performed). Although both play a role in directing drivers' gaze, most of the existing models for drivers' gaze prediction apply techniques developed for bottom-up saliency and do not consider influences of the drivers' actions explicitly. Likewise, common driving attention benchmarks lack relevant annotations for drivers' actions and the context in which they are performed. Therefore, to enable analysis and modeling of these factors for drivers' gaze prediction, we propose the following: 1) we correct the data processing pipeline used in DR(eye)VE to reduce noise in the recorded gaze data; 2) we then add per-frame labels for driving task and context; 3) we benchmark a number of baseline and SOTA models for saliency and driver gaze prediction and use new annotations to analyze how their performance changes in scenarios involving different tasks; and, lastly, 4) we develop a novel model that modulates drivers' gaze prediction with explicit action and context information. While reducing noise in the DR(eye)VE gaze data improves results of all models, we show that using task information in our proposed model boosts performance even further compared to bottom-up models on the cleaned up data, both overall (by 24% KLD and 89% NSS) and on scenarios that involve performing safety-critical maneuvers and crossing intersections (by up to 10--30% KLD). Extended annotations and code are available at <a class="link-external link-https" href="https://github.com/ykotseruba/SCOUT" rel="external noopener nofollow">this https URL</a>.

On the Use of Real-Time Multi-Object Detection and Classification for Automated Assessment of Operator Visual Attention Allocation

Classification of Safety Driver Attention During Autonomous Vehicle Operation

Utilising Visual Attention Cues for Vehicle Detection and Tracking

Multimodal Sensing and Computational Intelligence for Situation Awareness Classification in Autonomous Driving

Visual Detection of Personal Protective Equipment and Safety Gear on Industry Workers

Using Eye-Tracking Data to Predict Situation Awareness in Real Time During Takeover Transitions in Conditionally Automated Driving

Usage of eye tracker technology in examining attention distribution of operators of unmanned aerial vehicles

Automatic Visual Attention Detection for Mobile Eye Tracking Using Pre-Trained Computer Vision Models and Human Gaze

Learning Unsupervised Video Object Segmentation Through Visual Attention

A Real-time Assessment System for Cognitive Performances of Armored Vehicle Crews Based upon the Multi-source Information Fusion

Recognizing situation awareness of forklift operators based on eye-movement & EEG features

How Facial Features Convey Attention in Stationary Environments

Gaze-based Attention Recognition for Human-Robot Collaboration

Remote Sensing Target Detection Based on Visual Saliency Guidance and Classifier Fusion

Detecting Worker Attention Lapses in Human-Robot Interaction: An Eye Tracking and Multimodal Sensing Study

Human Visual Attention Prediction Boosts Learning & Performance of Autonomous Driving Agents

An Eye-Fixation Related Electroencephalography Technique for Predicting Situation Awareness: Implications for Driver State Monitoring Systems

Building Human Visual Attention Map for Construction Equipment Teleoperation

Understanding and Modeling the Effects of Task and Context on Drivers' Gaze Allocation

Paying Attention to Video Object Pattern Understanding

Learner Attentiveness and Engagement Analysis in Online Education Using Computer Vision