Abstract:Electroencephalography (EEG)-based emotion classification during music listening has gained increasing attention nowadays due to its promise of potential applications such as musical affective brain-computer interface (ABCI), neuromarketing, music therapy, and implicit multimedia tagging and triggering. However, music is an ecologically valid and complex stimulus that conveys certain emotions to listeners through compositions of musical elements. Using solely EEG signals to distinguish emotions remained challenging. This study aimed to assess the applicability of a multimodal approach by leveraging the EEG dynamics and acoustic characteristics of musical contents for the classification of emotional valence and arousal. To this end, this study adopted machine-learning methods to systematically elucidate the roles of the EEG and music modalities in the emotion modeling. The empirical results suggested that when whole-head EEG signals were available, the inclusion of musical contents did not improve the classification performance. The obtained performance of 74~76% using solely EEG modality was statistically comparable to that using the multimodality approach. However, if EEG dynamics were only available from a small set of electrodes (likely the case in real-life applications), the music modality would play a complementary role and augment the EEG results from around 61-67% in valence classification and from around 58-67% in arousal classification. The musical timber appeared to replace less-discriminative EEG features and led to improvements in both valence and arousal classification, whereas musical loudness was contributed specifically to the arousal classification. The present study not only provided principles for constructing an EEG-based multimodal approach, but also revealed the fundamental insights into the interplay of the brain activity and musical contents in emotion modeling.

EEG Processing in Emotion Recognition: Inspired from a Musical Staff.

An EEG-Based brain informatics application for enhancing music experience

Using Psychophysiologicalmeasures to Recognize Personalmusic Emotional Experience

Eeg-Based Emotion Recognition in Music Listening

Music emotion recognition based on temporal convolutional attention network using EEG

Music-evoked emotion recognition based on cognitive principles inspired EEG temporal and spectral features

EEG Emotion Recognition Applied to the Effect Analysis of Music on Emotion Changes in Psychological Healthcare

Electroencephalogram Emotion Recognition Based on Empirical Mode Decomposition and Optimal Feature Selection.

Enhancing Emotion Recognition Using Region-Specific Electroencephalogram Data and Dynamic Functional Connectivity

[An improved electroencephalogram feature extraction algorithm and its application in emotion recognition]

Electroencephalography Emotion Recognition Based on Rhythm Information Entropy Extraction

Music Emotion Recognition Through Sparse Canonical Correlation Analysis

Emotion Recognition and Regulation Based on Stacked Sparse Auto-Encoder Network and Personalized Reconfigurable Music

Music-evoked Emotion Classification Using EEG Correlation-Based Information

A review: Music-emotion recognition and analysis based on EEG signals

Fusion of electroencephalographic dynamics and musical contents for estimating emotional responses in music listening

EEG-driven automatic generation of emotive music based on transformer

Emotion Recognition from EEG Using Rhythm Synchronization Patterns with Joint Time-Frequency-Space Correlation.

Spatiotemporal Emotion Recognition Method Based on EEG Signals During Music Listening Using 1D-CNN & Stacked-LSTM.

EEG decoding for musical emotion with functional connectivity features

Emotion Recognition of EEG Signal Based on RMFS and PNN Algorithm