Abstract:Emotion recognition using the electroencephalogram (EEG) has garnered significant attention within the realm of human–computer interaction due to the wealth of genuine emotional data stored in EEG signals. However, traditional emotion recognition methods are deficient in mining the connection between multi-domain features and fitting their advantages. In this paper, we propose a novel capsule Transformer network based on a multi-domain feature for EEG-based emotion recognition, referred to as MES-CTNet. The model's core consists of a multichannel capsule neural network(CapsNet) embedded with ECA (Efficient Channel Attention) and SE (Squeeze and Excitation) blocks and a Transformer-based temporal coding layer. Firstly, a multi-domain feature map is constructed by combining the space–frequency–time characteristics of the multi-domain features as inputs to the model. Then, the local emotion features are extracted from the multi-domain feature maps by the improved CapsNet. Finally, the Transformer-based temporal coding layer is utilized to globally perceive the emotion feature information of the continuous time slices to obtain a final emotion state. The paper fully experimented on two standard datasets with different emotion labels, the DEAP and SEED datasets. On the DEAP dataset, MES-CTNet achieved an average accuracy of 98.31% in the valence dimension and 98.28% in the arousal dimension; it achieved 94.91% for the cross-session task on the SEED dataset, demonstrating superior performance compared to traditional EEG emotion recognition methods. The MES-CTNet method, utilizing a multi-domain feature map as proposed herein, offers a broader observation perspective for EEG-based emotion recognition. It significantly enhances the classification recognition rate, thereby holding considerable theoretical and practical value in the EEG emotion recognition domain.

ICaps-ResLSTM: Improved Capsule Network and Residual LSTM for EEG Emotion Recognition.

Multi-view Domain-Adaptive Representation Learning for EEG-based Emotion Recognition

Light-weight residual convolution-based capsule network for EEG emotion recognition

Emotion recognition from EEG based on multi-task learning with capsule network and attention mechanism

Multi-channel EEG-based emotion recognition via a multi-level features guided capsule network

EEG emotion recognition based on the attention mechanism and pre-trained convolution capsule network

Enhanced deep capsule network for EEG-based emotion recognition

TC-Net: A Transformer Capsule Network for EEG-based emotion recognition

Electroencephalogram Emotion Recognition Based on Empirical Mode Decomposition and Optimal Feature Selection.

Cascaded Convolutional Recurrent Neural Networks for EEG Emotion Recognition Based on Temporal–Frequency–Spatial Features

Using electroencephalogram for emotion recognition based on filter-bank long short-term memory networks

A model for electroencephalogram emotion recognition: Residual block-gated recurrent unit with attention mechanism

Deep Sparse Autoencoder and Recursive Neural Network for EEG Emotion Recognition

Bi-CapsNet: A Binary Capsule Network for EEG-Based Emotion Recognition

A Novel DE-CNN-BiLSTM Multi-Fusion Model for EEG Emotion Recognition

SST-CRAM: spatial-spectral-temporal based convolutional recurrent neural network with lightweight attention mechanism for EEG emotion recognition

An Efficient LSTM Network for Emotion Recognition from Multichannel EEG Signals

EEG-based Emotion Recognition via Channel-wise Attention and Self Attention

Spatial-temporal features-based EEG emotion recognition using graph convolution network and long short-term memory

Emotion recognition with convolutional neural network and EEG-based EFDMs

MES-CTNet: A Novel Capsule Transformer Network Base on a Multi-Domain Feature Map for Electroencephalogram-Based Emotion Recognition