Abstract:Recognition of a human's continuous emotional states in real time plays an important role in machine emotional intelligence and human-machine interaction. Existing real-time emotion recognition systems use stimuli with low ecological validity (e.g., picture, sound) to elicit emotions and to recognise only valence and arousal. To overcome these limitations, in this paper, we construct a standardised database of 16 emotional film clips that were selected from over one thousand film excerpts. Based on emotional categories that are induced by these film clips, we propose a real-time movie-induced emotion recognition system for identifying an individual's emotional states through the analysis of brain waves. Thirty participants took part in this study and watched 16 standardised film clips that characterise real-life emotional experiences and target seven discrete emotions and neutrality. Our system uses a 2-s window and a 50 percent overlap between two consecutive windows to segment the EEG signals. Emotional states, including not only the valence and arousal dimensions but also similar discrete emotions in the valence-arousal coordinate space, are predicted in each window. Our real-time system achieves an overall accuracy of 92.26 percent in recognising high-arousal and valenced emotions from neutrality and 86.63 percent in recognising positive from negative emotions. Moreover, our system classifies three positive emotions (joy, amusement, tenderness) with an average of 86.43 percent accuracy and four negative emotions (anger, disgust, fear, sadness) with an average of 65.09 percent accuracy. These results demonstrate the advantage over the existing state-of-the-art real-time emotion recognition systems from EEG signals in terms of classification accuracy and the ability to recognise similar discrete emotions that are close in the valence-arousal coordinate space.

Inter-Brain EEG Feature Extraction and Analysis for Continuous Implicit Emotion Tagging During Video Watching

Exploiting EEG signals and audiovisual feature fusion for video emotion recognition

Hybrid video emotional tagging using users’ EEG and video content

Identifying valence and arousal levels via connectivity between EEG channels

Exploring EEG Microstates for Affective Computing: Decoding Valence and Arousal Experiences During Video Watching

Multi-modal emotion analysis from facial expressions and electroencephalogram.

Using electroencephalogram for emotion recognition based on filter-bank long short-term memory networks

Real-Time Movie-Induced Discrete Emotion Recognition from EEG Signals

Electroencephalogram Emotion Recognition Based on Empirical Mode Decomposition and Optimal Feature Selection.

Multi-Target Positive Emotion Recognition from EEG Signals

Electroencephalography Amplitude Modulation Analysis for Automated Affective Tagging of Music Video Clips

Interpretable Emotion Classification Using Multidomain Feature of EEG Signals

Channel attention convolutional aggregation network based on video-level features for EEG emotion recognition

Content-Based Video Emotion Tagging Augmented by Users’ Multiple Physiological Responses

Facial Expression and EEG Fusion for Investigating Continuous Emotions of Deaf Subjects

Toward EEG-Based Objective Assessment of Emotion Intensity

Examining the Relationship between EEG Dynamics and Emotion Ratings during Video Watching using Adaptive Mixture Independent Component Analysis.

Implicit Hybrid Video Emotion Tagging By Integrating Video Content And Users' Multiple Physiological Responses

Implicit Video Emotion Tagging from Audiences’ Facial Expression

Emotion Feature Analysis and Recognition Based on Reconstructed EEG Sources

Multi-Domain Encoding of Spatiotemporal Dynamics in EEG for Emotion Recognition