Abstract:In human interactions, gaze may be used to acquire information for goal-directed actions, to acquire information related to the interacting partner's actions, and in the context of multimodal communication. At present, there are no models of gaze behavior in the context of vision that adequately incorporate these three components. In this study, we aimed to uncover and quantify patterns of within-person gaze-action coupling, gaze-gesture and gaze-speech coupling, and coupling between one person's gaze and another person's manual actions, gestures, or speech (or exogenous attraction of gaze) during dyadic collaboration. We showed that in the context of a collaborative Lego Duplo-model copying task, within-person gaze-action coupling is strongest, followed by within-person gaze-gesture coupling, and coupling between gaze and another person's actions. When trying to infer gaze location from one's own manual actions, gestures, or speech or that of the other person, only one's own manual actions were found to lead to better inference compared to a baseline model. The improvement in inferring gaze location was limited, contrary to what might be expected based on previous research. We suggest that inferring gaze location may be most effective for constrained tasks in which different manual actions follow in a quick sequence, while gaze-gesture and gaze-speech coupling may be stronger in unconstrained conversational settings or when the collaboration requires more negotiation. Our findings may serve as an empirical foundation for future theory and model development, and may further be relevant in the context of action/intention prediction for (social) robotics and effective human–robot interaction.

Gaze quality assisted automatic recognition of social contexts in collaborative Tetris

Using Dual Eye-Tracking to Unveil Coordination and Expertise in Collaborative Tetris

Exploring personal aspects using eye-tracking modality in Tetris-playing

3D Gaze Tracking for Studying Collaborative Interactions in Mixed-Reality Environments

Eyes on the Game: Deciphering Implicit Human Signals to Infer Human Proficiency, Trust, and Intent

Gaze-action coupling, gaze-gesture coupling, and exogenous attraction of gaze in dyadic interactions

Gaze in a real-world social interaction: A dual eye-tracking study

Gaze Coordination of Groups in Dynamic Events - A Tool to Facilitate Analyses of Simultaneous Gazes Within a Team

Detecting Gaze Towards Eyes in Natural Social Interactions and its Use in Child Assessment

Evaluating User Experience and Data Quality in Gamified Data Collection for Appearance-Based Gaze Estimation

Utilizing Interactive Surfaces to Enhance Learning, Collaboration and Engagement: Insights from Learners' Gaze and Speech

Deep neural networks for collaborative learning analytics: Evaluating team collaborations using student gaze point prediction

Gaze detection as a social cue to initiate natural human-robot collaboration in an assembly task

Gaze estimation learning architecture as support to affective, social and cognitive studies in natural human-robot interaction

CasualGaze: Towards Modeling and Recognizing Casual Gaze Behavior for Efficient Gaze-based Object Selection

EXPLORING GAZE BEHAVIOR TO ASSESS PERFORMANCE IN DIGITAL GAME-BASED LEARNING SYSTEMS

Social Visual Behavior Analytics for Autism Therapy of Children Based on Automated Mutual Gaze Detection

Multimodal Observation and Interpretation of Subjects Engaged in Problem Solving

Human Gaze and Head Rotation during Navigation, Exploration and Object Manipulation in Shared Environments with Robots

Charting the Silent Signals of Social Gaze: Automating Eye Contact Assessment in Face-to-Face Conversations