Abstract:It is crucial to evaluate learning outcomes by identifying the cognitive state of the learner during the learning process. Studies utilizing Electroencephalography (EEG) and other peripheral physiological signals, combined with deep learning models, have demonstrated improved performance in cognitive state recognition. These studies have primarily focused on unimodal data, which are vulnerable to various types of noise, making it difficult to fully capture and represent cognitive states. Leveraging the complementarity between multimodal physiological signals can mitigate the impact of anomalies in unimodal data, thereby improving the accuracy and stability of cognitive state recognition. Therefore, this study proposes a multimodal physiological signal feature representation fusion model based on multi-level attention (PSFMMA). The model aims to integrate multimodal physiological signals to identify learners' cognitive states with greater stability and accuracy. PSFMMA first extracts the temporal features of physiological signals by multiplexing the embedding layer. Subsequently, it generates signal representation vectors by further extracting semantic features through a signal feature mapping layer and enhancing important features with designed attention modules. Finally, the model employs an attention mechanism based on different signal representation vectors to fuse multimodal information for identifying learners' cognitive states. This study designs various learning activities and collects electroencephalography (EEG), electrodermal activity (EDA), and photoplethysmography (PPG) data from 22 participants engaging in these activities to create the Based on Learning Activities Collection (BLAC) dataset. The proposed model was evaluated on the BLAC dataset, achieving an identification accuracy of 96.32 ± 0.32%. The results demonstrate that the model can effectively recognize learners' cognitive states. Furthermore, the model's performance was validated on the publicly available emotion classification dataset DEAP, attaining an accuracy of 99.15 ± 0.12%. The source code is available at https://github.com/chengshudaxuesheng/PSFMMA.

Design and Implementation of the Reliable Learning Style Recognition Mechanism Based on Fusion Labels and Ensemble Classification

FGCM: Noisy Label Learning via Fine-Grained Confidence Modeling

Emotion Detection in Online Social Networks: A Multilabel Learning Approach

Learning Unit State Recognition Based on Multi-channel Data Fusion

Personality Recognition on Social Media with Label Distribution Learning

Design and Implementation of an EEG-Based Learning-Style Recognition Mechanism

The Adaptive Learning System Based on Learning Style and Cognitive State

A fuzzy classification approach for learning style prediction based on web mining technique in e-learning environments

A learning style classification approach based on deep belief network for large-scale online education

Online Professional-Creative Fusion Music Major Students' Classroom State Recognition Based on the Integration of DSC and LeNet-5 Models

Unsupervised Learning Style Classification for Learning Path Generation in Online Education Platforms

Learning Style Identification Using Semi-Supervised Self-Taught Labeling

Learning Resource Recommendation in E-Learning Systems Based on Online Learning Style

Online Learning Style Modeling for Course Recommendation

Multi-Source Multi-Label Learning for User Profiling in Online Games

TSMG: A Deep Learning Framework for Recognizing Human Learning Style Using EEG Signals

Research on dynamic learning style identification based on hybrid Neural Networks

Learner's cognitive state recognition based on multimodal physiological signal fusion

Enhanced learning resource recommendation based on online learning style model

Discriminable Multi-Label Attribute Selection for Pre-Course Student Performance Prediction

Representation-Enhanced Semi-Supervised Learning for Flotation Working Condition Recognition