Abstract:Eye movements have long been studied as a window into the attentional mechanisms of the human brain and made accessible as novelty style human-machine interfaces. However, not everything that we gaze upon, is something we want to interact with; this is known as the Midas Touch problem for gaze interfaces. To overcome the Midas Touch problem, present interfaces tend not to rely on natural gaze cues, but rather use dwell time or gaze gestures. Here we present an entirely data-driven approach to decode human intention for object manipulation tasks based solely on natural gaze cues. We run data collection experiments where 16 participants are given manipulation and inspection tasks to be performed on various objects on a table in front of them. The subjects' eye movements are recorded using wearable eye-trackers allowing the participants to freely move their head and gaze upon the scene. We use our Semantic Fovea, a convolutional neural network model to obtain the objects in the scene and their relation to gaze traces at every frame. We then evaluate the data and examine several ways to model the classification task for intention prediction. Our evaluation shows that intention prediction is not a naive result of the data, but rather relies on non-linear temporal processing of gaze cues. We model the task as a time series classification problem and design a bidirectional Long-Short-Term-Memory (LSTM) network architecture to decode intentions. Our results show that we can decode human intention of motion purely from natural gaze cues and object relative position, with $91.9\%$ accuracy. Our work demonstrates the feasibility of natural gaze as a Zero-UI interface for human-machine interaction, i.e., users will only need to act naturally, and do not need to interact with the interface itself or deviate from their natural eye movement patterns.

Design and Implementation of Human Intention Estimation System Based on Eye Movement Tracking

Development of an Eog (Electro-Oculography) Based Human-Computer Interface

An Electro-oculography Based Human-computer Interaction System

Application of Eye Tracking in Intelligent User Interface

Eye Tracking Based Control System for Natural Human-Computer Interaction

Eye-Tracking Technology and Its Application in e-Learning System

User intent perception by gesture and eye tracking

Research on Human-Computer Interaction Intention Recognition Based on EEG and Eye Movement

Design and Realization of Eye Control System for Small Ground Unmanned Platform

An Informatics System for Exploringeye Movements in Reading

Eye-Gaze Tracking Research Based on Image Processing

Eye-Tracking Monitoring Based on PMUT Arrays

3D Intuitive Gesture Interaction via Motion Sensing

Design and Implementation of A Reading Auxiliary Apparatus Based on Electrooculography

Eye tracking method based on mobile big data in computer environment

Eye Movement Recognition based Human Machine Interaction System

Play games with eyes: A portable EOG based HCI system for game control

A novel eye movement detection algorithm for EOG driven human computer interface

MIDAS: Deep learning human action intention prediction from natural eye movement patterns

Eye Tracking and Eye-Based Human–Computer Interaction

A real-time attitude recognition by eye-tracking