Abstract:Electroencephalography (EEG)-based emotion classification during music listening has gained increasing attention nowadays due to its promise of potential applications such as musical affective brain-computer interface (ABCI), neuromarketing, music therapy, and implicit multimedia tagging and triggering. However, music is an ecologically valid and complex stimulus that conveys certain emotions to listeners through compositions of musical elements. Using solely EEG signals to distinguish emotions remained challenging. This study aimed to assess the applicability of a multimodal approach by leveraging the EEG dynamics and acoustic characteristics of musical contents for the classification of emotional valence and arousal. To this end, this study adopted machine-learning methods to systematically elucidate the roles of the EEG and music modalities in the emotion modeling. The empirical results suggested that when whole-head EEG signals were available, the inclusion of musical contents did not improve the classification performance. The obtained performance of 74~76% using solely EEG modality was statistically comparable to that using the multimodality approach. However, if EEG dynamics were only available from a small set of electrodes (likely the case in real-life applications), the music modality would play a complementary role and augment the EEG results from around 61-67% in valence classification and from around 58-67% in arousal classification. The musical timber appeared to replace less-discriminative EEG features and led to improvements in both valence and arousal classification, whereas musical loudness was contributed specifically to the arousal classification. The present study not only provided principles for constructing an EEG-based multimodal approach, but also revealed the fundamental insights into the interplay of the brain activity and musical contents in emotion modeling.

Study on Linguistic Computing for Music Emotion

Linguistic computing model for music emotion

Music's Affective Computing Model Based on Fuzzy Logic

Research on music emotion cognition model based on linguistic value computing

Image–Music Synesthesia-Aware Learning Based on Emotional Similarity Recognition

Human-centric Music Medical Therapy Exploration System

Digital Empirical Research of Influencing Factors of Musical Emotion Classification Based on Pleasure-Arousal Musical Emotion Fuzzy Model

Using Psychophysiologicalmeasures to Recognize Personalmusic Emotional Experience

Emotion-Driven Chinese Folk Music-Image Retrieval Based on De-Svm

Emotional Recognition For Chime Bell Music

Research on Emotional Vocabulary-Driven Personalized Music Retrieval

User-Adaptive Music Emotion Recognition

Effects of Individual Factors on Perceived Emotion and Felt Emotion of Music: Based on Machine Learning Methods

Intelligent Analysis Of Music'S Affective Features And Fxpressive Pattern

Intelligent Analysis of Music's Affective Features and Expressive Pattern

Real-time Human-Music Emotional Interaction Based on Multimodal Analysis

Emotion Based Image Musicalization

Fusion of electroencephalographic dynamics and musical contents for estimating emotional responses in music listening

Real-Time Human-Music Emotional Interaction Based on Deep Learning and Multimodal Sentiment Analysis

A survey of music emotion recognition

EEG Emotion Recognition Applied to the Effect Analysis of Music on Emotion Changes in Psychological Healthcare