Abstract:Considerable attention has been paid to physiological signal-based emotion recognition in the field of affective computing. For reliability and user-friendly acquisition, electrodermal activity (EDA) has a great advantage in practical applications. However, EDA-based emotion recognition with large-scale subjects is still a tough problem. The traditional well-designed classifiers with hand-crafted features produce poorer results because of their limited representation abilities. And the deep learning models with auto feature extraction suffer the overfitting drop-off because of large-scale individual differences. Since music has a strong correlation with human emotion, static music can be involved as the external benchmark to constrain various dynamic EDA signals. In this article, we make an attempt by fusing the subject’s individual EDA features and the external evoked music features. And we propose an end-to-end multimodal framework, the one-dimensional residual temporal and channel attention network (RTCAN-1D). For EDA features, the channel-temporal attention mechanism for EDA-based emotion recognition is first involved in mine the temporal and channel-wise dynamic and steady features. The comparisons with single EDA-based SOTA models on DEAP and AMIGOS datasets prove the effectiveness of RTCAN-1D to mine EDA features. For music features, we simply process the music signal with the open-source toolkit openSMILE to obtain external feature vectors. We conducted systematic and extensive evaluations. The experiments on the current largest music emotion dataset PMEmo validate that the fusion of EDA and music is a reliable and efficient solution for large-scale emotion recognition.

A New Multilabel System for Automatic Music Emotion Recognition

A Efficient Multimodal Framework for Large Scale Emotion Recognition by Fusing Music and Electrodermal Activity Signals

Using Psychophysiologicalmeasures to Recognize Personalmusic Emotional Experience

Automatic Music Emotion Classification Using a New Classification Algorithm

A New Fuzzy Classifier For Music Emotion Based On Conditional Probability

Human-centric Music Medical Therapy Exploration System

Joint Learning of Emotions in Music and Generalized Sounds

Automatic music emotion classification model for movie soundtrack subtitling based on neuroscientific premises

A Comparison Study of Deep Learning Methodologies for Music Emotion Recognition

User-Adaptive Music Emotion Recognition

Digital Empirical Research of Influencing Factors of Musical Emotion Classification Based on Pleasure-Arousal Musical Emotion Fuzzy Model

Automatic ECG-Based Emotion Recognition in Music Listening

Real-time Human-Music Emotional Interaction Based on Multimodal Analysis

MMD-MII Model: A Multilayered Analysis and Multimodal Integration Interaction Approach Revolutionizing Music Emotion Classification

Study on Linguistic Computing for Music Emotion

Real-Time Human-Music Emotional Interaction Based on Deep Learning and Multimodal Sentiment Analysis

Visual-Audio Emotion Recognition Based on Multi-Task and Ensemble Learning with Multiple Features

A Multimodal Framework for Large-Scale Emotion Recognition by Fusing Music and Electrodermal Activity Signals

Symbolic & Acoustic: Multi-domain Music Emotion Modeling for Instrumental Music

Boosting for Multi-Modal Music Emotion Classification.

Emotion Recognition of Playing Musicians From EEG, ECG, and Acoustic Signals