Abstract:Considerable attention has been paid for physiological signal-based emotion recognition in field of affective computing. For the reliability and user friendly acquisition, Electrodermal Activity (EDA) has great advantage in practical applications. However, the EDA-based emotion recognition with hundreds of subjects still lacks effective solution. In this paper, our work makes an attempt to fuse the subject individual EDA features and the external evoked music features. And we propose an end-to-end multimodal framework, the 1-dimensional residual temporal and channel attention network (RTCAN-1D). For EDA features, the novel convex optimization-based EDA (CvxEDA) method is applied to decompose EDA signals into pahsic and tonic signals for mining the dynamic and steady features. The channel-temporal attention mechanism for EDA-based emotion recognition is firstly involved to improve the temporal- and channel-wise representation. For music features, we process the music signal with the open source toolkit openSMILE to obtain external feature vectors. The individual emotion features from EDA signals and external emotion benchmarks from music are fused in the classifing layers. We have conducted systematic comparisons on three multimodal datasets (PMEmo, DEAP, AMIGOS) for 2-classes valance/arousal emotion recognition. Our proposed RTCAN-1D outperforms the existing state-of-the-art models, which also validate that our work provides an reliable and efficient solution for large scale emotion recognition. Our code has been released at this https URL.

MediaEval 2014: THU-HCSIL Approach to Emotion in Music Task Using Multi-level Regression.

Multi-Scale Approaches to the MediaEval 2015 "emotion in Music" Task.

A Efficient Multimodal Framework for Large Scale Emotion Recognition by Fusing Music and Electrodermal Activity Signals

Using Psychophysiologicalmeasures to Recognize Personalmusic Emotional Experience

Learning Music Emotion Primitives via Supervised Dynamic Clustering.

Human-centric Music Medical Therapy Exploration System

Automatic Music Emotion Classification Using a New Classification Algorithm

A New Fuzzy Classifier For Music Emotion Based On Conditional Probability

A Deep Bidirectional Long Short-Term Memory Based Multi-Scale Approach for Music Dynamic Emotion Prediction

Emotional Recognition For Chime Bell Music

User-Adaptive Music Emotion Recognition

Svr Based Double-Scale Regression For Dynamic Emotion Prediction In Music

Beatsens' Solution for MediaEval 2014 Emotion in Music Task.

DBLSTM-based Multi-Scale Fusion for Dynamic Emotion Prediction in Music.

A Regression Approach to Categorical Music Emotion Recognition

Multi-scale Context Based Attention for Dynamic Music Emotion Prediction

Music emotion recognition based on middle and high level features

Automatic ECG-Based Emotion Recognition in Music Listening

Multimodal Music Emotion Recognition with Hierarchical Cross-Modal Attention Network

Visual-Audio Emotion Recognition Based on Multi-Task and Ensemble Learning with Multiple Features

Music Emotion Research Based on Reinforcement Learning and Multimodal Information