Abstract:Physiological signals can more realistically reflect human emotional states. To overcome the limitations imposed in single-modal emotion recognition, emotion recognition of multimodal physiological signals has received increasingly widespread attention. However, the original fusion models usually ignore the different distributions of multiple signals and how to capture complementary features from multimodal information effectively. This paper proposes an effective classification model for multimodal physiological signals to address the above issues based on modeling the heterogeneity and correlation between multimodal signals. First, differential entropy features are extracted from Electroencephalography (EEG) signals and peripheral physiological signals (PPS) such as Electrocardiographic (ECG) signals, Electromyographic (EMG) signals, and other physiological signals. Then, according to the different distributions and frequency characteristics of the acquired signals, the EEG signal features are made into a three-dimensional feature map and input to the neural network to extract the frequency spatial dimension features. Further temporal features are extracted from the peripheral physiological signals using a long and short-term memory network. Finally, the EEG and peripheral physiological signal features were fused and input to a multimodal long and short-term memory network to extract the association between different modalities and perform classification. The experiments were conducted on the benchmark DEAP dataset, and the results showed that the classification accuracy of the proposed model in this paper was 95.89% and 94.99% in the arousal dimension and the valence dimension, respectively, which were 2.77% and 3.11% higher compared to the unimodal EEG model, respectively. This paper also analyzed the effects of different peripheral physiological signals on emotion recognition.

Emotion Recognition from Physiological Signals Using Multi-Hypergraph Neural Networks.

Physiological Signals-based Emotion Recognition via High-order Correlation Learning

Emotion Recognition By Edge-Weighted Hypergraph Neural Network

EEG Emotion Recognition Based on Self-attention Dynamic Graph Neural Networks

A Multi-Head Pseudo Nodes Based Spatial–temporal Graph Convolutional Network for Emotion Perception from GAIT

Multimodal Physiological Signal Emotion Recognition Based on Convolutional Recurrent Neural Network

Emotion recognition based on multiple physiological signals

Personality-Aware Personalized Emotion Recognition from Physiological Signals.

Emotion extraction based on multi bio-signal using back-propagation neural network

EEG-Based Emotion Recognition Using Partial Directed Coherence Dense Graph Propagation

Correlation-Driven Multi-Modality Graph Decomposition for Cross-Subject Emotion Recognition

Multimodal Emotion Recognition Model using Physiological Signals

Multi-domain fusion deep graph convolution neural network for EEG emotion recognition

Personalized Emotion Recognition by Personality-Aware High-Order Learning of Physiological Signals

A Novel and Powerful Dual-Stream Multi-Level Graph Convolution Network for Emotion Recognition

Hierarchical emotion-recognition framework based on discriminative brain neural network topology and ensemble co-decision strategy

Effective Emotion Recognition by Learning Discriminative Graph Topologies in EEG Brain Networks

Multi-channel EEG emotion recognition through residual graph attention neural network

Multidimensional Feature in Emotion Recognition Based on Multi-Channel EEG Signals.

Minimum spanning tree based graph neural network for emotion classification using EEG

Emotion Recognition from Physiological Channels Using Graph Neural Network