Abstract:Emotion charting using multimodal signals has gained great demand for stroke-affected patients, for psychiatrists while examining patients, and for neuromarketing applications. Multimodal signals for emotion charting include electrocardiogram (ECG) signals, electroencephalogram (EEG) signals, and galvanic skin response (GSR) signals. EEG, ECG, and GSR are also known as physiological signals, which can be used for identification of human emotions. Due to the unbiased nature of physiological signals, this field has become a great motivation in recent research as physiological signals are generated autonomously from human central nervous system. Researchers have developed multiple methods for the classification of these signals for emotion detection. However, due to the non-linear nature of these signals and the inclusion of noise, while recording, accurate classification of physiological signals is a challenge for emotion charting. Valence and arousal are two important states for emotion detection; therefore, this paper presents a novel ensemble learning method based on deep learning for the classification of four different emotional states including high valence and high arousal (HVHA), low valence and low arousal (LVLA), high valence and low arousal (HVLA) and low valence high arousal (LVHA). In the proposed method, multimodal signals (EEG, ECG, and GSR) are preprocessed using bandpass filtering and independent components analysis (ICA) for noise removal in EEG signals followed by discrete wavelet transform for time domain to frequency domain conversion. Discrete wavelet transform results in spectrograms of the physiological signal and then features are extracted using stacked autoencoders from those spectrograms. A feature vector is obtained from the bottleneck layer of the autoencoder and is fed to three classifiers SVM (support vector machine), RF (random forest), and LSTM (long short-term memory) followed by majority voting as ensemble classification. The proposed system is trained and tested on the AMIGOS dataset with k-fold cross-validation. The proposed system obtained the highest accuracy of 94.5% and shows improved results of the proposed method compared with other state-of-the-art methods.

Arousal-Valence Classification from Peripheral Physiological Signals Using Long Short-Term Memory Networks

Arousal and Valence Classification Model Based on Long Short-Term Memory and DEAP Data for Mental Healthcare Management

An Efficient Machine Learning-Based Emotional Valence Recognition Approach Towards Wearable EEG

A Comparative Study of Machine Learning Techniques for Emotion Recognition from Peripheral Physiological Signals

End-to-End Emotion Recognition using Peripheral Physiological Signals

Temporal Convolutional Network-Enhanced Real-Time Implicit Emotion Recognition with an Innovative Wearable fNIRS-EEG Dual-Modal System

Decoding Emotional Experience through Physiological Signal Processing

An Emotion Recognition System Based on Physiological Signals Obtained by Wearable Sensors

WEARS: Wearable Emotion AI with Real-time Sensor data

Emotion Recognition from ECG Signals Using Wavelet Scattering and Machine Learning

CorrNet: Fine-Grained Emotion Recognition for Video Watching Using Wearable Physiological Sensors

Emotion and Stress Recognition Utilizing Galvanic Skin Response and Wearable Technology: A Real-time Approach for Mental Health Care

An Ensemble Learning Method for Emotion Charting Using Multimodal Physiological Signals

CNN and LSTM-Based Emotion Charting Using Physiological Signals

Automated Affective Computing Based on Bio-Signals Analysis and Deep Learning Approach

Behavioral and Physiological Signals-Based Deep Multimodal Approach for Mobile Emotion Recognition

Automatic Emotion Recognition Using Temporal Multimodal Deep Learning

Online Learning for Wearable EEG-Based Emotion Classification

Valence-Arousal Model based Emotion Recognition using EEG, peripheral physiological signals and Facial Expression

Emotion recognition based on multiple physiological signals

Emotion Recognition Using Electrodermal Activity Signals and Multiscale Deep Convolution Neural Network