Abstract:Electroencephalography (EEG)-based emotion classification during music listening has gained increasing attention nowadays due to its promise of potential applications such as musical affective brain-computer interface (ABCI), neuromarketing, music therapy, and implicit multimedia tagging and triggering. However, music is an ecologically valid and complex stimulus that conveys certain emotions to listeners through compositions of musical elements. Using solely EEG signals to distinguish emotions remained challenging. This study aimed to assess the applicability of a multimodal approach by leveraging the EEG dynamics and acoustic characteristics of musical contents for the classification of emotional valence and arousal. To this end, this study adopted machine-learning methods to systematically elucidate the roles of the EEG and music modalities in the emotion modeling. The empirical results suggested that when whole-head EEG signals were available, the inclusion of musical contents did not improve the classification performance. The obtained performance of 74~76% using solely EEG modality was statistically comparable to that using the multimodality approach. However, if EEG dynamics were only available from a small set of electrodes (likely the case in real-life applications), the music modality would play a complementary role and augment the EEG results from around 61-67% in valence classification and from around 58-67% in arousal classification. The musical timber appeared to replace less-discriminative EEG features and led to improvements in both valence and arousal classification, whereas musical loudness was contributed specifically to the arousal classification. The present study not only provided principles for constructing an EEG-based multimodal approach, but also revealed the fundamental insights into the interplay of the brain activity and musical contents in emotion modeling.

Digital Empirical Research of Influencing Factors of Musical Emotion Classification Based on Pleasure-Arousal Musical Emotion Fuzzy Model

Using Psychophysiologicalmeasures to Recognize Personalmusic Emotional Experience

A New Fuzzy Classifier For Music Emotion Based On Conditional Probability

Human-centric Music Medical Therapy Exploration System

Music's Affective Computing Model Based on Fuzzy Logic

Research on music emotion cognition model based on linguistic value computing

Linguistic computing model for music emotion

Study on Linguistic Computing for Music Emotion

Effects of Individual Factors on Perceived Emotion and Felt Emotion of Music: Based on Machine Learning Methods

Music Emotion Research Based on Reinforcement Learning and Multimodal Information

Emotional Recognition For Chime Bell Music

Machine Learning–based Modeling and Prediction of the Intrinsic Relationship Between Human Emotion and Music

User-Adaptive Music Emotion Recognition

Study On The Affective Property Of Music

An artificial intelligence-based classifier for musical emotion expression in media education

Automatic Recognition of Emotion in Music

Computer Analysis and Automatic Recognition Technology of Music Emotion

Fusion of electroencephalographic dynamics and musical contents for estimating emotional responses in music listening

Physiological State Can Help Predict the Perceived Emotion of Music: Evidence from ECG and EDA Signals

Musical Melody and Emotional Evocation - An Empirical Study from Statistical Analysis

The Analysis Of The Music Emotion Based On The Pleasure-Arousal-Dominance Model