Boundaries in the eyes: measure event segmentation during naturalistic video watching using eye tracking

Jiashen Li,Zhengyue Cheng,Xin Hao,Wei Liu

DOI: https://doi.org/10.1101/2024.08.22.609279

2024-08-23

Abstract:During naturalistic information processing, individuals spontaneously segment their continuous experiences into discrete events, a phenomenon known as event segmentation. Traditional methods for assessing this process, which include subjective reports and neuroimaging techniques, often disrupt real-time segmentation or are costly and time intensive. Our study investigated the potential of measuring event segmentation by recording and analyzing eye movements while participants view naturalistic videos. We collected eye movement data from healthy young adults as they watched commercial films (N=104), or online Science, Technology, Engineering, and Mathematics (STEM) educational courses (N=44). We analyzed changes in pupil size and eye movement speed near event boundaries and employed inter-subject correlation analysis (ISC) and hidden Markov models (HMM) to identify patterns indicative of event segmentation. We observed that both the speed of eye movements and pupil size dynamically responded to event boundaries, exhibiting heightened sensitivity to high-strength boundaries. Our analyses further revealed that event boundaries synchronized eye movements across participants. These boundaries, can be effectively identified by HMM, yielded higher within-event similarity values and aligned with human-annotated boundaries. Importantly, HMM-based event segmentation metrics responded to experimental manipulations and predicted learning outcomes. This study provided a comprehensive computational framework for measuring event segmentation using eye-tracking. With the widespread accessibility of low-cost eye-tracking devices, the ability to measure event segmentation from eye movement data promises to deepen our understanding of this process in diverse real-world settings.

Neuroscience

What problem does this paper attempt to address?

The paper attempts to address the issue of how to use eye-tracking technology to measure event segmentation during natural video viewing. Traditional methods such as subjective reports and neuroimaging techniques have issues with interference or high costs when measuring event segmentation in real-time. Therefore, the researchers aim to explore a new, easy-to-implement method to identify event segmentation by analyzing participants' eye movement data while watching natural videos. Specifically, they use changes in pupil size and eye movement speed as basic indicators, combined with advanced computational methods such as Inter-Subject Correlation (ISC) and Hidden Markov Model (HMM) to identify event boundaries. Additionally, the study examines the response of these eye-tracking indicators to experimental manipulations and their predictive power for memory performance. The ultimate goal is to determine a reliable eye-tracking signature for quantifying the event segmentation process in different natural video stimuli.