Abstract:Academic emotions refer to various emotional experiences in connection with learners’ academic activities while learning, and these emotions are vital to the development of learners’ physiology and mentality. Facial expression recognition (FER) technology has been widely used in online learning to identify learners’ academic emotions. However, learners often inadvertently cover part of their face with their hands during online learning, which affects the accuracy of the technology's recognition of academic emotions. Most existing databases lack facial expression data with hand occlusion, which makes it difficult for researchers to further improve recognition accuracy. Therefore, this research established an online learners’ facial expression database with hand occlusion (OLFED-HO) to solve the above problem. This database has a total of 92,947 facial expression images of online learners, including four different hand occlusion situations (no occlusion, left occlusion, middle occlusion, and right occlusion) and seven academic emotions (confusion, curiosity, distraction, enjoyment, fatigue, depression, and neutrality). Then, to indicate the high reliability of the database established in this study, we analyzed the confusion matrix and concluded that the expression labels marked by different external coders have a high internal consistency. The database is expected to further promote the application of expression recognition technology in the field of education and provide online learners’ facial expressions with hand occlusion for the academic emotion database. In addition, an automatic facial expression recognition method with transfer learning based on region attention networks (RAN) is proposed in this paper, which efficiently reduces the impact of hand occlusion. The proposed architecture achieves an accuracy of 89% on the test set of our database.

Study on Learner's Facial Expression Recognition

Facial Expression Study Based on 3D Facial Emotion Recognition

Automatic Recognition and Analysis of Academic Emotions Based on Facial Expressions during Online Learning Environments

The Exploration of Facial Expression Recognition in Distance Education Learning System.

Using Facial Expression to Detect Emotion in E-learning System: A Deep Learning Method.

Emotion Analysis Based on Facial Expression Recognition in Smart Learning Environment

Implementation of an adaptive E-learning platform with facial emotion recognition

A Deep Learning Approach for Online Learning Emotion Recognition

Evaluation of Online Teaching Quality Based on Facial Expression Recognition

Inferring Academic Emotion In Online Learning Based On Spontaneous Facial Expression

Deep Convolutional Neural Networks with Curriculum Learning for Facial Expression Recognition

Learning Status Recognition Method Based on Facial Expressions in e-Learning

Spontaneous facial expression database for academic emotion inference in online learning.

Deep Learning Based Emotion Recognition and Visualization of Figural Representation

A Teacher and Student Facial Expression Recognition Model Based on Classroom Teaching Videos

Classifying Emotions and Engagement in Online Learning Based on a Single Facial Expression Recognition Neural Network

Studies Advanced in Development and Application of Facial Expression Recognition

Advances and Application of Facial Expression and Learning Emotion Recognition in Classroom

Spontaneous Visual Database for Detecting Learning-Centered Emotions During Online Learning

Spontaneous facial expression database of learners’ academic emotions in online learning with hand occlusion

Multi-modal Facial Expression Feature Based on Deep-Neural Networks