Abstract:Human–computer interaction (HCI) plays a significant role in modern education, and emotion recognition is essential in the field of HCI. The potential of emotion recognition in education remains to be explored. Confusion is the primary cognitive emotion during learning and significantly affects student engagement. Recent studies show that electroencephalogram (EEG) signals, obtained through electrodes placed on the scalp, are valuable for studying brain activity and identifying emotions. In this paper, we propose a fusion framework for confusion analysis in learning based on EEG signals, combining feature extraction and temporal self-attention. This framework capitalizes on the strengths of traditional feature extraction and deep-learning techniques, integrating local time-frequency features and global representation capabilities. We acquire localized time-frequency features by partitioning EEG samples into time slices and extracting Power Spectral Density (PSD) features. We introduce the Transformer architecture to capture the comprehensive EEG characteristics and utilize a multi-head self-attention mechanism to extract the global dependencies among the time slices. Subsequently, we employ a classification module based on a fully connected layer to classify confusion emotions accurately. To assess the effectiveness of our method in the educational cognitive domain, we conduct thorough experiments on a public dataset CAL, designed for confusion analysis during the learning process. In both subject-dependent and subject-independent experiments, our method attained an accuracy/F1 score of 90.94%/0.94 and 66.08%/0.65 for the binary classification task and an accuracy/F1 score of 87.59%/0.87 and 41.28%/0.41 for the four-class classification task. It demonstrated superior performance and stronger generalization capabilities than traditional machine learning classifiers and end-to-end methods. The evidence demonstrates that our proposed framework is effective and feasible in recognizing cognitive emotions.

An EEG-based attention recognition method: fusion of time domain, frequency domain, and non-linear dynamics features

Dual Attentive Fusion for EEG-based Brain-Computer Interfaces

Multi-Level Attention Recognition of EEG Based on Feature Selection

HASTF: a hybrid attention spatio-temporal feature fusion network for EEG emotion recognition

EEG-based attention recognition

A Real-Time EEG-based BCI System for Attention Recognition in Ubiquitous Environment

A New Classifier for the Recognition of EEG Patterns Associated to Attention Tasks

Improving EEG-based Emotion Recognition by Fusing Time-frequency And Spatial Representations

Multimodal EEG and Eye Tracking Feature Fusion Approaches for Attention Classification in Hybrid BCIs

Classification of Auditory Attention Focuses During Speech Perception.

Visual Attention Recognition Based on Nonlinear Dynamical Parameters of EEG.

A Hybrid Method Fusing Frequency Recognition with Attention Detection to Enhance an Asynchronous Brain-Computer Interface

Improved Classification Performance of EEG-fNIRS Multimodal Brain-Computer Interface Based on Multi-Domain Features and Multi-Level Progressive Learning

A Fusion Framework for Confusion Analysis in Learning Based on EEG Signals

EEG Emotion Recognition Based on Frequency and Channel Convolutional Attention

Temporal attention fusion network with custom loss function for EEG--fNIRS classification

Decoding Attentional State to Faces and Scenes Using EEG Brainwaves

Comparative Study of Attention-Related Features on Attention Monitoring Systems with a Single EEG Channel

Fusing Frequency-Domain Features and Brain Connectivity Features for Cross-Subject Emotion Recognition

Improving Intention Detection in Single-Trial Classification through Fusion of EEG and Eye-tracker Data

EEG Emotion Recognition Using an Attention Mechanism Based on an Optimized Hybrid Model