Abstract:Lack of supervision is a particularly challenging problem in E-learning or distance learning environments. A wide range of research efforts and technologies have been explored to alleviate its impact by monitoring students engagement, such as emotion or learning behaviors. However, the current research still lacks multi-dimensional computational measures for analyzing learners engagement from the interactions that occur in digital learning environment. In this paper, we propose an integrated framework to identify learning engagement from three facets: affect, behavior and cognitive state, which are conveyed by learners facial expressions, eye movement behaviors and the overall performance during short video learning session. To recognize the three states of learners, three channel data is recorded: 1) video/image sequence captured by camera; 2) eye movement information from a non-intrusive and cost-effective eye tracker; and 3) click stream data from mouse. Based on these modalities, a multi-channel data fusion strategy is designed that concatenates time series features of three channels in the same time segment to predict course learning performance. We also presented a new method to make the self-reported annotations more reliable without using external observers verification. To validate the approach and methods, 46 participants were invited to attend a representative on-line course that consists of short videos in our designed learning environment. The results demonstrated the effectiveness of the proposed framework and methods in monitoring learning engagement. More importantly, a prototype system was developed to detect learners emotional and eye behavioral engagement in real-time as well as predict the learning performance of learners after they had completed each short video course.

An Eye Detection Based Multi-modal &Amp;#x0022;e-Supervisor&#x0022; in Distant Education

An Attention Detection System Based on Gaze Estimation Using Self-supervised Learning

Research and implementation of supervised learning system based on attention detection in E-learning environment

Multimodal human attention detection for reading from facial expression, eye gaze, and mouse dynamics

Multi-Sensor Eye-Tracking Systems and Tools for Capturing Student Attention and Understanding Engagement in Learning: A Review

Automated Alertness and Emotion Detection for Empathic Feedback During E-Learning

Eye-Tracking Technology and Its Application in e-Learning System

Learner Attentiveness and Engagement Analysis in Online Education Using Computer Vision

An Online Education Effect Assessing Method Based on YOLOv8 and Vision Transformer

Learning State Assessment in Online Education Based on Multiple Facial Features Detection

An Eye-tracking Experiment Research on Three-part-separated Screen Web Course

Recognizing Multidimensional Engagement of E-Learners Based on Multi-Channel Data in E-Learning Environment.

Student Engagement Detection Using Emotion Analysis, Eye Tracking and Head Movement with Machine Learning

An English video teaching classroom attention evaluation model incorporating multimodal information

Automatic Detection of Mind Wandering Using Mobile Device in the Scenario of Online and Mobile Education

Combining Facial Behavioral Cues, Eye Movements and EEG-Based Attention to Improve Prediction of Reading Failure

An Approach of Query Audience's Attention in Virtual Speech

Modeling Students' Attention in the Classroom using Eyetrackers

A multimodal facial cues based engagement detection system in e-learning context using deep learning approach

Design and Implementation of A Reading Auxiliary Apparatus Based on Electrooculography

Detection of Dynamic Changes of Electrodermal Activity to Predict the Classroom Performance of College Students

An Eye Detection Based Multi-modal &Amp;#x0022;e-Supervisor&amp;#x0022; in Distant Education