Abstract:In recent years, emotion recognition based on electroencephalography (EEG) signals has attracted plenty of attention. Most of the existing works focused on normal or depressed people. Due to the lack of hearing ability, it is difficult for hearing-impaired people to express their emotions through language in their social activities. In this work, we collected the EEG signals of hearing-impaired subjects when they were watching six kinds of emotional video clips (happiness, inspiration, neutral, anger, fear, and sadness) for emotion recognition. The biharmonic spline interpolation method was utilized to convert the traditional frequency domain features, Differential Entropy (DE), Power Spectral Density (PSD), and Wavelet Entropy (WE) into the spatial domain. The patch embedding (PE) method was used to segment the feature map into the same patch to obtain the differences in the distribution of emotional information among brain regions. For feature classification, a compact residual network with Depthwise convolution (DC) and Pointwise convolution (PC) is proposed to separate spatial and channel mixing dimensions to better extract information between channels. Dependent subject experiments based on 70% training sets and 30% testing sets were performed. The results showed that the average classification accuracies by PE (DE), PE (PSD), and PE (WE) were 91.75%, 85.53%, and 75.68%, respectively which were improved by 11.77%, 23.54%, and 16.61% compared with DE, PSD, and WE. Moreover, the comparison experiments were carried out on the SEED and DEAP datasets with PE (DE), which achieved average accuracies of 90.04% (positive, neutral, and negative) and 88.75% (high valence and low valence). By exploring the emotional brain regions, we found that the frontal, parietal, and temporal lobes of hearing-impaired people were associated with emotional activity compared to normal people whose main emotional brain area was the frontal lobe.

Sonification for EEG Frequency Spectrum and EEG-Based Emotion Features.

Deep Spectrum Feature Representations for Speech Emotion Recognition

An EEG-Based brain informatics application for enhancing music experience

Exploiting EEG signals and audiovisual feature fusion for video emotion recognition

Electroencephalogram Emotion Recognition Based on Empirical Mode Decomposition and Optimal Feature Selection.

Electroencephalography Emotion Recognition Based on Rhythm Information Entropy Extraction

EEG decoding for musical emotion with functional connectivity features

Predicting Emotions Perceived from Sounds

Emotion Recognition of Subjects With Hearing Impairment Based on Fusion of Facial Expression and EEG Topographic Map

Emotion recognition from sound stimuli based on back-propagation neural networks and electroencephalograms

EEG-Based Emotion Recognition for Hearing Impaired and Normal Individuals With Residual Feature Pyramids Network Based on Time–Frequency–Spatial Features

Research Progress of EEG-Based Emotion Recognition: A Survey

HASTF: a hybrid attention spatio-temporal feature fusion network for EEG emotion recognition

[An improved electroencephalogram feature extraction algorithm and its application in emotion recognition]

A Hybrid Fuzzy Cognitive Map/Support Vector Machine Approach for EEG-Based Emotion Classification Using Compressed Sensing

Improving EEG-based Emotion Recognition by Fusing Time-frequency And Spatial Representations

Emotion recognition with residual network driven by spatial-frequency characteristics of EEG recorded from hearing-impaired adults in response to video clips

EEG-based Cognitive Load Estimation of Acoustic Parameters for Data Sonification

EEG emotion recognition approach using multi-scale convolution and feature fusion

EEG emotion recognition based on TQWT-features and hybrid convolutional recurrent neural network

EEG-driven automatic generation of emotive music based on transformer