Abstract:Deep learning has recently achieved remarkable success in emotion recognition based on Electroencephalogram (EEG), in which convolutional neural networks (CNNs) are the mostly used models. However, due to the local feature learning mechanism, CNNs have difficulty in capturing the global contextual information involving temporal domain, frequency domain, intra-channel and inter-channel. In this paper, we propose a Transformer Capsule Network (TC-Net), which mainly contains an EEG Transformer module to extract EEG features and an Emotion Capsule module to refine the features and classify the emotion states. In the EEG Transformer module, EEG signals are partitioned into non-overlapping windows. A Transformer block is adopted to capture global features among different windows, and we propose a novel patch merging strategy named EEG-PatchMerging (EEG-PM) to better extract local features. In the Emotion Capsule module, each channel of the EEG feature maps is encoded into a capsule to better characterize the spatial relationships among multiple features. Experimental results on two popular datasets ( i.e. , DEAP and DREAMER) demonstrate that the proposed method achieves the state-of-the-art performance in the subject-dependent scenario. Specifically, on DEAP (DREAMER), our TC-Net achieves the average accuracies of 98.76% (98.59%), 98.81% (98.61%) and 98.82% (98.67%) at valence, arousal and dominance dimensions, respectively. Moreover, the proposed TC-Net also shows high effectiveness in multi-state emotion recognition tasks using the popular VA and VAD models. The main limitation of the proposed model is that it tends to obtain relatively low performance in the cross-subject recognition task, which is worthy of further study in the future.

A Transformer Convolutional Network With the Method of Image Segmentation for EEG-Based Emotion Recognition

JDAT: Joint-Dimension-Aware Transformer with Strong Flexibility for EEG Emotion Recognition

CIT-EmotionNet: CNN Interactive Transformer Network for EEG Emotion Recognition

EEG-based Emotion Recognition Via Transformer Neural Architecture Search

Transformers for EEG-Based Emotion Recognition: A Hierarchical Spatial Information Learning Model

Temporal-spatial Representation Learning Transformer for EEG-based Emotion Recognition

ERTNet: an interpretable transformer-based framework for EEG emotion recognition

Emotion Classification Based on Transformer and CNN for EEG Spatial–Temporal Feature Learning

A spatiotemporal symmetrical transformer structure for EEG emotion recognition

Spatiotemporal Gated Graph Transformer for EEG-Based Emotion Recognition

TPRO-NET: an EEG-based emotion recognition method reflecting subtle changes in emotion

Spatial-temporal Transformers for EEG Emotion Recognition

TC-Net: A Transformer Capsule Network for EEG-based emotion recognition

EEG emotion recognition using attention-based convolutional transformer neural network

Using electroencephalogram for emotion recognition based on filter-bank long short-term memory networks

EmT: A Novel Transformer for Generalized Cross-subject EEG Emotion Recognition

Electroencephalogram Emotion Recognition Based on Empirical Mode Decomposition and Optimal Feature Selection.

EEG-Based Emotion Recognition by Convolutional Neural Network with Multi-Scale Kernels

A 3D Convolutional Neural Network for Emotion Recognition Based on EEG Signals.

EEG emotion recognition approach using multi-scale convolution and feature fusion

AMDET: Attention based Multiple Dimensions EEG Transformer for Emotion Recognition