Abstract:Physiological signals can more realistically reflect human emotional states. To overcome the limitations imposed in single-modal emotion recognition, emotion recognition of multimodal physiological signals has received increasingly widespread attention. However, the original fusion models usually ignore the different distributions of multiple signals and how to capture complementary features from multimodal information effectively. This paper proposes an effective classification model for multimodal physiological signals to address the above issues based on modeling the heterogeneity and correlation between multimodal signals. First, differential entropy features are extracted from Electroencephalography (EEG) signals and peripheral physiological signals (PPS) such as Electrocardiographic (ECG) signals, Electromyographic (EMG) signals, and other physiological signals. Then, according to the different distributions and frequency characteristics of the acquired signals, the EEG signal features are made into a three-dimensional feature map and input to the neural network to extract the frequency spatial dimension features. Further temporal features are extracted from the peripheral physiological signals using a long and short-term memory network. Finally, the EEG and peripheral physiological signal features were fused and input to a multimodal long and short-term memory network to extract the association between different modalities and perform classification. The experiments were conducted on the benchmark DEAP dataset, and the results showed that the classification accuracy of the proposed model in this paper was 95.89% and 94.99% in the arousal dimension and the valence dimension, respectively, which were 2.77% and 3.11% higher compared to the unimodal EEG model, respectively. This paper also analyzed the effects of different peripheral physiological signals on emotion recognition.

The Relationship Between Chaotic Characteristics of Physiological Signals and Emotion Based on Approximate Entropy

Application of Chaos Characteristics about Physiological Signals in Emotion Recognition Based on Approximate Entropy

Electroencephalography Emotion Recognition Based on Rhythm Information Entropy Extraction

Changes of Permutation Pattern Entropy and Ordinal Pattern Entropy During Three Emotion States: Natural, Happiness and Sadness

Finding Needles in a Haystack: Recognizing Emotions Just from Your Heart

Dynamic Entropy-Based Pattern Learning to Identify Emotions from EEG Signals Across Individuals

Automated Emotion Identification Using Fourier–Bessel Domain-Based Entropies

An Emotion Recognition System Based on Physiological Signals Obtained by Wearable Sensors

Emotion Recognition Using Empirical Mode Decomposition and Approximation Entropy

Ensemble Emotion Recognizing with Multiple Modal Physiological Signals

A Domain Generalization and Residual Network Based Emotion Recognition from Physiological Signals

Emotion Recognition Based on the Multiple Physiological Signals

Recent Survey on Emotion Recognition Using Physiological Signals

Application Of Permutation Entropy And Permutation Min-Entropy In Multiple Emotional States Analysis Of Rri Time Series

Emotion recognition based on multiple physiological signals

Analysis of affective ECG signals toward emotion recognition

Mixed Emotion Recognition Based on EEG Signals

EEG-Based Emotion Recognition by Exploiting Fused Network Entropy Measures of Complex Networks Across Subjects.

Using physiological signals for emotion recognition

Physiological Signal Processing for Emotional Feature Extraction.

Analysis of Physiological for Emotion Recognition with the IRS Model