Emotion Classification Based on Transformer and CNN for EEG Spatial–Temporal Feature Learning

Xiuzhen Yao,Tianwen Li,Peng Ding,Fan Wang,Lei Zhao,Anmin Gong,Wenya Nan,Yunfa Fu
DOI: https://doi.org/10.3390/brainsci14030268
IF: 3.333
2024-03-12
Brain Sciences
Abstract:Objectives: The temporal and spatial information of electroencephalogram (EEG) signals is crucial for recognizing features in emotion classification models, but it excessively relies on manual feature extraction. The transformer model has the capability of performing automatic feature extraction; however, its potential has not been fully explored in the classification of emotion-related EEG signals. To address these challenges, the present study proposes a novel model based on transformer and convolutional neural networks (TCNN) for EEG spatial–temporal (EEG ST) feature learning to automatic emotion classification. Methods: The proposed EEG ST-TCNN model utilizes position encoding (PE) and multi-head attention to perceive channel positions and timing information in EEG signals. Two parallel transformer encoders in the model are used to extract spatial and temporal features from emotion-related EEG signals, and a CNN is used to aggregate the EEG's spatial and temporal features, which are subsequently classified using Softmax. Results: The proposed STT-CNN model achieved an accuracy of 96.67% on the SEED dataset and accuracies of 95.73%, 96.95%, and 96.34% for the arousal–valence, arousal, and valence dimensions, respectively, for the DEAP dataset. Conclusions: The results demonstrate the effectiveness of the proposed ST-TCNN model, with superior performance in emotion classification compared to recent relevant studies. Significance: The proposed STT-CNN model has the potential to be used for EEG-based automatic emotion recognition.
neurosciences
What problem does this paper attempt to address?
The problem this paper attempts to address is: In emotion classification based on electroencephalogram (EEG) signals, existing methods overly rely on manual feature extraction, leading to the loss of valuable information and thus affecting the model's learning ability. To overcome this challenge, this study proposes a new model based on Transformer and Convolutional Neural Network (CNN) (EEG ST-TCNN) for automatically extracting spatial-temporal features of EEG to achieve more accurate emotion classification. Specifically, this study aims to: 1. **Automatic Feature Extraction**: Utilize the Transformer model to automatically extract spatial and temporal features from EEG signals, reducing the information loss caused by manual feature extraction. 2. **Improve Classification Accuracy**: Enhance the accuracy of emotion classification by combining the advantages of Transformer and CNN. 3. **Validate the Model's Effectiveness**: Conduct experiments on two public datasets, SEED and DEAP, to validate the effectiveness and superiority of the proposed EEG ST-TCNN model. Through these objectives, this study hopes to provide a more efficient and accurate method for EEG-based emotion recognition.