Abstract:Considerable attention has been paid for physiological signal-based emotion recognition in field of affective computing. For the reliability and user friendly acquisition, Electrodermal Activity (EDA) has great advantage in practical applications. However, the EDA-based emotion recognition with hundreds of subjects still lacks effective solution. In this paper, our work makes an attempt to fuse the subject individual EDA features and the external evoked music features. And we propose an end-to-end multimodal framework, the 1-dimensional residual temporal and channel attention network (RTCAN-1D). For EDA features, the novel convex optimization-based EDA (CvxEDA) method is applied to decompose EDA signals into pahsic and tonic signals for mining the dynamic and steady features. The channel-temporal attention mechanism for EDA-based emotion recognition is firstly involved to improve the temporal- and channel-wise representation. For music features, we process the music signal with the open source toolkit openSMILE to obtain external feature vectors. The individual emotion features from EDA signals and external emotion benchmarks from music are fused in the classifing layers. We have conducted systematic comparisons on three multimodal datasets (PMEmo, DEAP, AMIGOS) for 2-classes valance/arousal emotion recognition. Our proposed RTCAN-1D outperforms the existing state-of-the-art models, which also validate that our work provides an reliable and efficient solution for large scale emotion recognition. Our code has been released at this https URL.

COSMIC: Music Emotion Recognition Combining Structure Analysis and Modal Interaction

Multimodal Music Emotion Recognition with Hierarchical Cross-Modal Attention Network

Human-centric Music Medical Therapy Exploration System

Linguistic computing model for music emotion

A Efficient Multimodal Framework for Large Scale Emotion Recognition by Fusing Music and Electrodermal Activity Signals

Image–Music Synesthesia-Aware Learning Based on Emotional Similarity Recognition

Mind Band: A Crossmedia AI Music Composing Platform

Study on Linguistic Computing for Music Emotion

Emotional Recognition For Chime Bell Music

Symbolic & Acoustic: Multi-domain Music Emotion Modeling for Instrumental Music

Multimodel Music Emotion Recognition Using Unsupervised Deep Neural Networks

Music's Affective Computing Model Based on Fuzzy Logic

Real-time Human-Music Emotional Interaction Based on Multimodal Analysis

MMD-MII Model: A Multilayered Analysis and Multimodal Integration Interaction Approach Revolutionizing Music Emotion Classification

Multimodal Emotion Recognition by Extracting Common and Modality-Specific Information.

Coordinate Embedding Transformer Model for Optical Music Recognition on Monophonic Scores

Research on Music Emotional Expression Based on Reinforcement Learning and Multimodal Information

Real-Time Human-Music Emotional Interaction Based on Deep Learning and Multimodal Sentiment Analysis

Multimodal Music Mood Classification by Fusion of Audio and Lyrics.

Constructing a Multimodal Music Teaching Model in College by Integrating Emotions

Bimodal Emotion Recognition Model for Minnan Songs