Abstract:In recent years, emotion recognition based on electroencephalography (EEG) signals has attracted plenty of attention. Most of the existing works focused on normal or depressed people. Due to the lack of hearing ability, it is difficult for hearing-impaired people to express their emotions through language in their social activities. In this work, we collected the EEG signals of hearing-impaired subjects when they were watching six kinds of emotional video clips (happiness, inspiration, neutral, anger, fear, and sadness) for emotion recognition. The biharmonic spline interpolation method was utilized to convert the traditional frequency domain features, Differential Entropy (DE), Power Spectral Density (PSD), and Wavelet Entropy (WE) into the spatial domain. The patch embedding (PE) method was used to segment the feature map into the same patch to obtain the differences in the distribution of emotional information among brain regions. For feature classification, a compact residual network with Depthwise convolution (DC) and Pointwise convolution (PC) is proposed to separate spatial and channel mixing dimensions to better extract information between channels. Dependent subject experiments based on 70% training sets and 30% testing sets were performed. The results showed that the average classification accuracies by PE (DE), PE (PSD), and PE (WE) were 91.75%, 85.53%, and 75.68%, respectively which were improved by 11.77%, 23.54%, and 16.61% compared with DE, PSD, and WE. Moreover, the comparison experiments were carried out on the SEED and DEAP datasets with PE (DE), which achieved average accuracies of 90.04% (positive, neutral, and negative) and 88.75% (high valence and low valence). By exploring the emotional brain regions, we found that the frontal, parietal, and temporal lobes of hearing-impaired people were associated with emotional activity compared to normal people whose main emotional brain area was the frontal lobe.

Emotion recognition from sound stimuli based on back-propagation neural networks and electroencephalograms

Emotion Recognition under Sound Stimuli Based on BP Neural Network and EEG

Enhancing Emotion Recognition Using Region-Specific Electroencephalogram Data and Dynamic Functional Connectivity

Electroencephalogram Emotion Recognition Based on Empirical Mode Decomposition and Optimal Feature Selection.

EMOTION RECOGNITION BASED ON REPRESENTATION DISSIMILARITY MATRIX

Music-evoked emotion recognition based on cognitive principles inspired EEG temporal and spectral features

Emotion extraction based on multi bio-signal using back-propagation neural network

Emotion recognition with residual network driven by spatial-frequency characteristics of EEG recorded from hearing-impaired adults in response to video clips

Emotion Recognition from Sound Events Based on Facial Expression Recognition

Improving Emotion Analysis For Speech-Induced Eegs Through Eemd-Hht-Based Feature Extraction And Electrode Selection

Eeg-Based Emotion Recognition in Music Listening

Music emotion recognition based on temporal convolutional attention network using EEG

EEG-Based Emotion Recognition for Hearing Impaired and Normal Individuals With Residual Feature Pyramids Network Based on Time–Frequency–Spatial Features

An EEG-Based Brain-Computer Interface for Emotion Recognition.

[An improved electroencephalogram feature extraction algorithm and its application in emotion recognition]

Design and Implementation of an Emotion Analysis System Based on EEG Signals

Image-Evoked Emotion Recognition for Hearing-Impaired Subjects with EEG Signals

Emotion Recognition Based on Low-Cost In-Ear EEG.

Emotion Classification with EEG Responses Evoked by Emotional Prosody of Speech

Neural Correlates of Perceived Emotions in Human Insula and Amygdala for Auditory Emotion Recognition

EEG Processing in Emotion Recognition: Inspired from a Musical Staff.