Abstract:Emotion recognition is a key component of human–computer interaction technology, for which facial electromyogram (fEMG) is an important physiological modality. Recently, deep-learning-based emotion recognition using fEMG signals has drawn increased attention. However, the ability of effective feature extraction and the demand of large-scale training data are two dominant factors that restrict the performance of emotion recognition. In this paper, a novel spatio-temporal deep forest (STDF) model is proposed to classify three categories of discrete emotions (neutral, sadness, and fear) using multi-channel fEMG signals. The feature extraction module fully extracts effective spatio-temporal features of fEMG signals using a combination of 2D frame sequences and multi-grained scanning. Meanwhile, a cascade forest-based classifier is designed to provide optimal structures for different scales of training data via automatically adjusting the number of cascade layers. The proposed model and five comparison methods were evaluated on our in-house fEMG dataset that included three discrete emotions and three channels of fEMG electrodes with a total of twenty-seven subjects. Experimental results demonstrate that the proposed STDF model achieves the best recognition performance with an average accuracy of 97.41%. Besides, our proposed STDF model can reduced the scale of training data to 50% while the average accuracy of emotion recognition is only reduced by about 5%. Our proposed model offers an effective solution for practical applications of fEMG-based emotion recognition.

Emotion Recognition Using Hidden Markov Models From Facial Temperature Sequence

Physiological Signals Based Emotional State Recognition Model of Audience

Exploring Spatio-Temporal Representations by Integrating Attention-based Bidirectional-LSTM-RNNs and FCNs for Speech Emotion Recognition

Emotion Recognition from Thermal Infrared Images Using Deep Boltzmann Machine

Facial Expression Recognition from Infrared Thermal Videos.

Multi-modal emotion analysis from facial expressions and electroencephalogram.

Facial Expression Recognition from Infrared Thermal Images Using Temperature Difference by Voting

Recognition of fearful emotion based on facial infrared thermal images

Spatio-temporal deep forest for emotion recognition based on facial electromyography signals

Real-Time facial expression recognition system based on HMM and feature point localization

Non-contact Emotion Analysis Based on Videos.

Spatio-temporal Analysis for Infrared Facial Expression Recognition from Videos.

Using electroencephalogram for emotion recognition based on filter-bank long short-term memory networks

Early Facial Expression Recognition Using Hidden Markov Models

Posed and spontaneous expression distinguishment from infrared thermal images

Modelling Temporal Information Using Discrete Fourier Transform for Recognizing Emotions in User-generated Videos

A Frown-Based Thermal Comfort Detection Method of Facial Emotion Recognition

Audio-visual emotion recognition based on Hidden Markov Model

EEG-based Emotion Estimation Using Adaptive Tracking of Discriminative Frequency Components

A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference

Speech emotion recognition using hidden Markov models