Abstract:Eye contact is a crucial element of non-verbal communication that signifies interest, attention, and participation in social interactions. As a result, measures of eye contact arise in a variety of applications such as the assessment of the social communication skills of children at risk for developmental disorders such as autism, or the analysis of turn-taking and social roles during group meetings. However, the automated measurement of visual attention during naturalistic social interactions is challenging due to the difficulty of estimating a subject's looking direction from video. This paper proposes a novel approach to eye contact detection during adult-child social interactions in which the adult wears a point-of-view camera which captures an egocentric view of the child's behavior. By analyzing the child's face regions and inferring their head pose we can accurately identify the onset and duration of the child's looks to their social partner's eyes. We introduce the Pose-Implicit CNN, a novel deep learning architecture that predicts eye contact while implicitly estimating the head pose. We present a fully automated system for eye contact detection that solves the sub-problems of end-to-end feature learning and pose estimation using deep neural networks. To train our models, we use a dataset comprising 22 hours of 156 play session videos from over 100 children, half of whom are diagnosed with Autism Spectrum Disorder. We report an overall precision of 0.76, recall of 0.80, and an area under the precision-recall curve of 0.79, all of which are significant improvements over existing methods.

Everyday Eye Contact Detection Using Unsupervised Gaze Target Discovery

A Single-Camera Gaze Tracking System under Natural Light

Accurate and Robust Eye Contact Detection During Everyday Mobile Device Interactions

Detecting Gaze Towards Eyes in Natural Social Interactions and its Use in Child Assessment

MutualEyeContact: A conversation analysis tool with focus on eye contact

Charting the Silent Signals of Social Gaze: Automating Eye Contact Assessment in Face-to-Face Conversations

Interested Object Detection Based on Gaze Using Low-cost Remote Eye Tracker

Training a Camera to Perform Long-Distance Eye Tracking by Another Eye-Tracker

Low-cost Geometry-based Eye Gaze Detection using Facial Landmarks Generated through Deep Learning

An Attention Detection System Based on Gaze Estimation Using Self-supervised Learning

Dual Attention Guided Gaze Target Detection in the Wild

Gaze Target Estimation inspired by Interactive Attention

Indicating eye contacts in one-to-many video teleconference with one web camera

CasualGaze: Towards Modeling and Recognizing Casual Gaze Behavior for Efficient Gaze-based Object Selection

It's Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation

Learning to Find Eye Region Landmarks for Remote Gaze Estimation in Unconstrained Settings

A Survey on Gaze Estimation

Are you really looking at me? A Feature-Extraction Framework for Estimating Interpersonal Eye Gaze from Conventional Video

EM-Gaze: Eye Context Correlation and Metric Learning for Gaze Estimation.

Quantifying Gaze Behavior During Real-World Interactions Using Automated Object, Face, and Fixation Detection

Gaze-Aided Eye Detection Via Appearance Learning