Abstract:It is crucial to evaluate learning outcomes by identifying the cognitive state of the learner during the learning process. Studies utilizing Electroencephalography (EEG) and other peripheral physiological signals, combined with deep learning models, have demonstrated improved performance in cognitive state recognition. These studies have primarily focused on unimodal data, which are vulnerable to various types of noise, making it difficult to fully capture and represent cognitive states. Leveraging the complementarity between multimodal physiological signals can mitigate the impact of anomalies in unimodal data, thereby improving the accuracy and stability of cognitive state recognition. Therefore, this study proposes a multimodal physiological signal feature representation fusion model based on multi-level attention (PSFMMA). The model aims to integrate multimodal physiological signals to identify learners' cognitive states with greater stability and accuracy. PSFMMA first extracts the temporal features of physiological signals by multiplexing the embedding layer. Subsequently, it generates signal representation vectors by further extracting semantic features through a signal feature mapping layer and enhancing important features with designed attention modules. Finally, the model employs an attention mechanism based on different signal representation vectors to fuse multimodal information for identifying learners' cognitive states. This study designs various learning activities and collects electroencephalography (EEG), electrodermal activity (EDA), and photoplethysmography (PPG) data from 22 participants engaging in these activities to create the Based on Learning Activities Collection (BLAC) dataset. The proposed model was evaluated on the BLAC dataset, achieving an identification accuracy of 96.32 ± 0.32%. The results demonstrate that the model can effectively recognize learners' cognitive states. Furthermore, the model's performance was validated on the publicly available emotion classification dataset DEAP, attaining an accuracy of 99.15 ± 0.12%. The source code is available at https://github.com/chengshudaxuesheng/PSFMMA.

A LSTM Network-based Learners’ Monitoring Model for Academic Self-efficacy Evaluation Using EEG Signal Analysis

High-Performance Personalized Heartbeat Classification Model for Long-Term ECG Signal

Emotional Monitoring of Learners Based on EEG Signal Recognition

M4Eeg: Matching Network-Based Mental Health Status Assessment Model Using Eeg Signals

Improve Affective Learning with EEG Approach

A Mental Health Detection Model Of College Students Under The Behavior Of Learning Burnout

Combining Facial Behavioral Cues, Eye Movements and EEG-Based Attention to Improve Prediction of Reading Failure

Evaluating Children’s Engagement Through Wrist-worn Biosensors and LSTM Network

LSTM Model with Self-Attention Mechanism for EEG Based Cross-Subject Fatigue Detection

Detecting the Confusion of Students in Massive Open Online Courses Using EEG

Learner's cognitive state recognition based on multimodal physiological signal fusion

Detection of Dynamic Changes of Electrodermal Activity to Predict the Classroom Performance of College Students

Using Support Vector Machine on EEG Signals for College Students' Immersive Learning Evaluation

Deep Learning-Based Assessment Model for Real-Time Identification of Visual Learners Using Raw EEG

Efficacy of Measuring Engagement during Computer-Based Training with Low-Cost Electroencephalogram (EEG) Sensor Outputs

Personalized Early Warning of Learning Performance for College Students: A Multi-level Approach Via Cognitive Ability and Learning State Modeling

Deep Learning-Based Signal Quality Assessment in Wearable ECG Monitoring.

Research on Education Big Data for Students Academic Performance Analysis based on Machine Learning

Prediction of Cognitive Load from Electroencephalography Signals Using Long Short-Term Memory Network

Application of Electroencephalography Sensors and Artificial Intelligence in Automated Language Teaching

EEG-Based Evaluation of Aesthetic Experience Using BiLSTM Network