Abstract:In the realm of education, feedback emerges as a pivotal component, serving to foster engagement and interaction while also facilitating the refinement of teaching methods to capture and maintain student attention. Traditional classroom assessment methods often struggle to accurately gauge the degree of comprehension among students during lectures, relying on manual comment collection that inherently carries the risk of inaccuracies. In response to this challenge, a novel system has been proposed, harnessing the power of Facial Emotion Recognition (FER) technology to capture student feedback. Within this framework, students are given a unique avenue to convey their emotions and reactions, employing facial expressions and gestures as the means to communicate. This innovative approach enables the analysis of students' emotional responses and thereby provides invaluable insights into their comprehension levels, as well as the overall quality and engagement experienced during lectures. The approach takes shape through the utilization of Computer Vision techniques, with a particular focus on an unobtrusive methodology for assessing students' overall engagement. Overcoming limitations of traditional assessment, our approach integrates compound scaling, employing the proposed Multitask EfficientNetB0 model recognized for its proved accuracy in emotion recognition (95.7%) and behavior analysis (96.3%) across diverse datasets (DAiSEE, iSED, iSAFFE). The behavioral classification system categorizes students into "Engaged" and "Disengaged" classes within a multi-class framework, providing nuanced insights into comprehension and Student engagement. Assessment metrics, including ROC Curves, Precision, Recall, and F1-Score, ensure a thorough evaluation. Our system's adaptability is demonstrated across varied educational environments, showcasing real-world efficacy in classrooms, laboratories, and seminar halls. The inclusion of MTCNN enhances face detection capabilities, facilitating robust analysis in dynamic scenarios. Expanding its applicability, the model has been put to the test in a range of educational settings, including classrooms, laboratory environments, and seminar halls, offering dual-capability analysis of both emotions and behavior. This comprehensive approach yields nuanced insights into student engagement and interaction, and its performance has been validated through real-world deployment within classrooms and seminars

The Syncretic Effect of Dual-Source Data on Affective Computing in Online Learning Contexts: A Perspective From Convolutional Neural Network With Attention Mechanism

Automatic Depression Prediction Via Cross-Modal Attention-Based Multi-Modal Fusion in Social Networks

Automatic Recognition and Analysis of Academic Emotions Based on Facial Expressions during Online Learning Environments

Temporal Convolutional Network-Enhanced Real-Time Implicit Emotion Recognition with an Innovative Wearable fNIRS-EEG Dual-Modal System

Combining Facial Expressions and Electroencephalography to Enhance Emotion Recognition

A Framework of Online Learning and Experiment System Based on Affective Computing

Utilizing Deep Learning Towards Multi-modal Bio-sensing and Vision-based Affective Computing

3-D Facial Expression Recognition via Attention-Based Multichannel Data Fusion Network

A Novel Adaptive Affective Cognition Analysis Model for College Students Using a Deep Convolution Neural Network and Deep Features

Using Facial Expression to Detect Emotion in E-learning System: A Deep Learning Method.

Research on Emotion Recognition for Online Learning in a Novel Computing Model

Constructing multi-modal emotion recognition model based on convolutional neural network

Facial Expression and Peripheral Physiology Fusion to Decode Individualized Affective Experience

A Dual Attention-based Modality-Collaborative Fusion Network for Emotion Recognition

Affective computing model for natural interaction based on large-scale self-built dataset

An Optimized Residual Network For Emotion Recognition Based On A Multi-Features Fusion Technology And Electroencephalography Signals

Multitask EfficientNet affective computing for student engagement detection

Evaluation and analysis of visual perception using attention-enhanced computation in multimedia affective computing

Learning multi-level representations for affective image recognition

Analysis of English classroom teaching behavior and strategies under adaptive deep learning under cognitive psychology

A Fusion Framework for Confusion Analysis in Learning Based on EEG Signals