Abstract:To further advance driver monitoring and assistance systems, it is important to understand how drivers allocate their attention, in other words, where do they tend to look and why. Traditionally, factors affecting human visual attention have been divided into bottom-up (involuntary attraction to salient regions) and top-down (driven by the demands of the task being performed). Although both play a role in directing drivers' gaze, most of the existing models for drivers' gaze prediction apply techniques developed for bottom-up saliency and do not consider influences of the drivers' actions explicitly. Likewise, common driving attention benchmarks lack relevant annotations for drivers' actions and the context in which they are performed. Therefore, to enable analysis and modeling of these factors for drivers' gaze prediction, we propose the following: 1) we correct the data processing pipeline used in DR(eye)VE to reduce noise in the recorded gaze data; 2) we then add per-frame labels for driving task and context; 3) we benchmark a number of baseline and SOTA models for saliency and driver gaze prediction and use new annotations to analyze how their performance changes in scenarios involving different tasks; and, lastly, 4) we develop a novel model that modulates drivers' gaze prediction with explicit action and context information. While reducing noise in the DR(eye)VE gaze data improves results of all models, we show that using task information in our proposed model boosts performance even further compared to bottom-up models on the cleaned up data, both overall (by 24% KLD and 89% NSS) and on scenarios that involve performing safety-critical maneuvers and crossing intersections (by up to 10--30% KLD). Extended annotations and code are available at <a class="link-external link-https" href="https://github.com/ykotseruba/SCOUT" rel="external noopener nofollow">this https URL</a>.

Visual attention computation in video of driving environment

A Fusion Method Aiming at Environmental Perception of Autonomous Vehicle Based on Visual Scheme

&Lt;title>automatic Traffic Real-Time Analysis System Based on Video</title>

Where and What: Driver Attention-based Object Detection

Object-Level Attention Prediction for Drivers in the Information-Rich Traffic Environment

DADA-2000: Can Driving Accident be Predicted by Driver Attention? Analyzed by A Benchmark

Spatial and Temporal Visual Attention Prediction in Videos Using Eye Movement Data

Image Visual Attention Computation and Application Via the Learning of Object Attributes

Visual attention for behavioral cloning in autonomous driving

Human Visual Attention Prediction Boosts Learning & Performance of Autonomous Driving Agents

Toward Driving Scene Understanding: A Paradigm and Benchmark Dataset for Ego-Centric Traffic Scene Graph Representation

An Image-based Approach of Task-driven Driving Scene Categorization

Understanding and Modeling the Effects of Task and Context on Drivers' Gaze Allocation

Utilising Visual Attention Cues for Vehicle Detection and Tracking

Seeing Beyond Views: Multi-View Driving Scene Video Generation with Holistic Attention

Exploring the Visual Attention Mechanism of Long-Distance Driving in an Underground Construction Cavern: Eye-Tracking and Simulated Driving

Brain Inspired Cognitive Model with Attention for Self-Driving Cars

Multisource Adaption for Driver Attention Prediction in Arbitrary Driving Scenes

Recent advancements in driver's attention prediction

Detecting Driver Cognition Alertness State from Visual Activities in Normal and Emergency Scenarios.

Driver Attention Tracking and Analysis