Abstract:The interaction between the vestibular and ocular system has primarily been studied in controlled environments. Consequently, off-the shelf tools for categorization of gaze events (e.g. fixations, pursuits, saccade) fail when head movements are allowed. Our approach was to collect a novel, naturalistic, and multimodal dataset of eye+head movements when subjects performed everyday tasks while wearing a mobile eye tracker equipped with an inertial measurement unit and a 3D stereo camera. This Gaze-in-the-Wild dataset (GW) includes eye+head rotational velocities (deg/s), infrared eye images and scene imagery (RGB+D). A portion was labelled by coders into gaze motion events with a mutual agreement of 0.72 sample based Cohen's $\kappa$. This labelled data was used to train and evaluate two machine learning algorithms, Random Forest and a Recurrent Neural Network model, for gaze event classification. Assessment involved the application of established and novel event based performance metrics. Classifiers achieve $\sim$90$\%$ human performance in detecting fixations and saccades but fall short (60$\%$) on detecting pursuit movements. Moreover, pursuit classification is far worse in the absence of head movement information. A subsequent analysis of feature significance in our best-performing model revealed a reliance upon absolute eye and head velocity, indicating that classification does not require spatial alignment of the head and eye tracking coordinate systems. The GW dataset, trained classifiers and evaluation metrics will be made publicly available with the intention of facilitating growth in the emerging area of head-free gaze event classification.

EDVAM: a 3D eye-tracking dataset for visual attention modeling in a virtual museum

An Eye-Tracking Dataset for Visual Attention Modelling in a Virtual Museum Context.

Learning Stereoscopic Visual Attention Model for 3d Video

PerimetryNet: A Multiscale Fine Grained Deep Network for Three-Dimensional Eye Gaze Estimation Using Visual Field Analysis

Benchmark 3D eye-tracking dataset for visual saliency prediction on stereoscopic 3D video

EHTask: Recognizing User Tasks from Eye and Head Movements in Immersive Virtual Reality

VEnvision3D: A Synthetic Perception Dataset for 3D Multi-Task Model Research

FixationNet: Forecasting Eye Fixations in Task-Oriented Virtual Environments.

Temporal continuity of visual attention for future gaze prediction in immersive virtual reality.

The Visual Experience Dataset: Over 200 Recorded Hours of Integrated Eye Movement, Odometry, and Egocentric Video

A Visual Attentive Model for Discovering Patterns in Eye-Tracking Data—A Proposal in Cultural Heritage

Audio-visual Saliency Prediction for Movie Viewing in Immersive Environments: Dataset and Benchmarks

GazeBaseVR, a large-scale, longitudinal, binocular eye-tracking dataset collected in virtual reality

Gaze-in-wild: A dataset for studying eye and head coordination in everyday activities

Panonut360: A Head and Eye Tracking Dataset for Panoramic Video

Fantastic Answers And Where To Find Them: Immersive Question-Directed Visual Attention

OpenEDS: Open Eye Dataset

Spatial and Temporal Visual Attention Prediction in Videos Using Eye Movement Data

Tasks Reflected in the Eyes: Egocentric Gaze-Aware Visual Task Type Recognition in Virtual Reality

3D-ARM-Gaze: a public dataset of 3D Arm Reaching Movements with Gaze information in virtual reality

3D Pop-Ups: Omnidirectional image visual saliency prediction based on crowdsourced eye-tracking data in VR