Abstract:Emotion recognition is crucial for human–computer interaction, and electroencephalography (EEG) stands out as a valuable tool for capturing and reflecting human emotions. In this study, we propose a hierarchical hybrid model called Mixed Attention-based Convolution and Transformer Network (MACTN). This model is designed to collectively capture both local and global temporal information and is inspired by insights from neuroscientific research on the temporal dynamics of emotions. First, we introduce depth-wise temporal convolution and separable convolution to extract local temporal features. Then, a self-attention-based transformer is used to integrate the sparse global emotional features. Besides, channel attention mechanism is designed to identify the most task-relevant channels, facilitating the capture of relationships between different channels and emotional states. Extensive experiments are conducted on three public datasets under both offline and online evaluation modes. In the multi-class cross-subject online evaluation using the THU-EP dataset, MACTN demonstrates an approximate 8% enhancement in 9-class emotion recognition accuracy in comparison to state-of-the-art methods. In the multi-class cross-subject offline evaluation using the DEAP and SEED datasets, a comparable performance is achieved solely based on the raw EEG signals, without the need for prior knowledge or transfer learning during the feature extraction and learning process. Furthermore, ablation studies have shown that integrating self-attention and channel-attention mechanisms improves classification performance. This method won the Emotional BCI Competition's final championship in the World Robot Contest. The source code is available at https://github.com/ThreePoundUniverse/MACTN .

EEG-based Emotion Recognition Via Transformer Neural Architecture Search

JDAT: Joint-Dimension-Aware Transformer with Strong Flexibility for EEG Emotion Recognition

EEG Emotion Recognition Based on Self-attention Dynamic Graph Neural Networks

Temporal-spatial Representation Learning Transformer for EEG-based Emotion Recognition

EmT: A Novel Transformer for Generalized Cross-subject EEG Emotion Recognition

EEG-based Emotion Recognition via Neural Architecture Search

A Transformer Convolutional Network With the Method of Image Segmentation for EEG-Based Emotion Recognition

Transformers for EEG-Based Emotion Recognition: A Hierarchical Spatial Information Learning Model

A spatiotemporal symmetrical transformer structure for EEG emotion recognition

Spatial-temporal Transformers for EEG Emotion Recognition

TC-Net: A Transformer Capsule Network for EEG-based emotion recognition

Emotion Classification Based on Transformer and CNN for EEG Spatial–Temporal Feature Learning

ERTNet: an interpretable transformer-based framework for EEG emotion recognition

EEG emotion recognition using attention-based convolutional transformer neural network

CIT-EmotionNet: CNN Interactive Transformer Network for EEG Emotion Recognition

Spatiotemporal Gated Graph Transformer for EEG-Based Emotion Recognition

Electroencephalogram Emotion Recognition Based on Empirical Mode Decomposition and Optimal Feature Selection.

Temporal Aware Mixed Attention-based Convolution and Transformer Network (MACTN) for EEG Emotion Recognition

Temporal aware Mixed Attention-based Convolution and Transformer Network for cross-subject EEG emotion recognition

TPRO-NET: an EEG-based emotion recognition method reflecting subtle changes in emotion

Multi-Source Domain Adaptation with Transformer-based Feature Generation for Subject-Independent EEG-based Emotion Recognition