A Social Robot As Your Reading Companion: Exploring the Relationships Between Gaze Patterns and Knowledge Gains

Xuan Liu,Jiachen Ma,Qiang Wang
DOI: https://doi.org/10.1007/s12193-023-00418-5
2024-01-01
Journal on Multimodal User Interfaces
Abstract:Intelligent tutoring systems have been widely used in educational activities over the past 20 years. With significantly less effort put into writing or reading assistance, the majority of intelligent tutoring systems focus on mathematics or problem solving activities. However, with the development of E-reading-centered E-education, how to improve students’ learning performance during reading has become increasingly important. Therefore, in this paper, we take a first step in the direction of an adaptive intelligent tutoring system by investigating how different reading strategies relate to knowledge gain based on gaze features and how an embodied social robot affects gaze patterns and reading strategies. The findings showed that different knowledge gains have significant differences in scanning methods and reading depth, and that the feedback given by social robots significantly affects participants’ gaze patterns during the whole reading process. To automatically differentiate between two levels of knowledge gain, several prediction experiments based on various reading strategy-related gaze features were carried out. The results demonstrate that saccades are the best predictors of knowledge gain, with the best model having an average accuracy of 74.2%. Finally, real-time simulation experiments were conducted with sixty participants using the leave-one-out method, and an accurate prediction of the level of knowledge gain of 71. 5% was achieved.
What problem does this paper attempt to address?