Abstract:Emotions are a complex multi-faceted phenomenon. To assess the complexity of emotions from different facets, multi-modal approaches are necessary. However, multi-modal approaches are rarely used for assessing emotions, especially in the context of computer programming. This study adopts a multi-modal approach to understand the changes in students’ perception of emotions before and after working on programming problems. Understanding these changes in students’ perceptions may enable educators to devise interventions that help students adjust their perceptions and regulate their emotions as per their skills. We conducted a one-on-one programming session and retrospective think-aloud interview with seventeen students from an introductory programming course. During the programming session, students filled surveys and performed four programming tasks. While working on these tasks, students’ eye gaze, video of face and screen, and electrodermal activity data were also collected using a non-invasive device. The data collection for this study was multi-modal, with a mix of both qualitative and quantitative data collection methods. Data analysis was primarily qualitative, with additional triangulation of qualitative and biometric data for select exemplars. The findings of this study suggest that students experience changes in emotions because of many reasons, for instance, they encountered repeated errors, they set high standards for their performance, or they could not manage time. For some students, negative emotions changed to positive emotions when they solved errors without any external help or achieved more than what they expected going into the task. Moreover, the triangulation of qualitative and biometric data of two participants provides a fine-grained analysis of their emotions and behaviors and confirmed the change in the perception of their emotions while performing the programming tasks. Emotions, Multi-modal data, and Programming

Towards Emotion-awareness in Programming Education with Behavior-based Emotion Estimation

Real-Time Classroom Behavior Analysis for Enhanced Engineering Education: An AI-Assisted Approach

Prototype of Educational Affective Arousal Evaluation System Based on Facial and Speech Emotion Recognition

Towards affective learning with an EEG feedback approach.

Automatic Recognition and Analysis of Academic Emotions Based on Facial Expressions during Online Learning Environments

Detect Students' Academic Emotions In Classroom: Measurement, Self-Perception And Manifested Behaviors

Emotion generation method in online physical education teaching based on data mining of teacher-student interactions

Student Emotion Recognition Using Computer Vision as an Assistive Technology for Education

Real-time EEG-based Emotion Monitoring Using Stable Features

How Do First-Year Engineering Students’ Emotions Change While Working on Programming Problems?

An Emotional Interactive Algorithm Based on Affective Computing in Distance Education

Automated Alertness and Emotion Detection for Empathic Feedback During E-Learning

Realization of Self-Adaptive Higher Teaching Management Based Upon Expression and Speech Multimodal Emotion Recognition

A Framework of Online Learning and Experiment System Based on Affective Computing

Emotion Recognition of Students Based on Facial Expressions in Online Education Based on the Perspective of Computer Simulation

A FCM Based Approach for Emotion Prediction in Educational Game

OPTES: A Tool for Behavior-based Student Programming Progress Estimation.

Artificial Intelligence Technology-Driven Teacher Mental State Assessment and Improvement Method

SensEmo: Enabling Affective Learning through Real-time Emotion Recognition with Smartwatches

Facial Expression Recognition for Probing Students’ Emotional Engagement in Science Learning

Learner's Emotion Prediction in Smart Learning Environment