Abstract:Exploring proper way to conduct multi-speech feature fusion for cross-corpus speech emotion recognition is crucial as different speech features could provide complementary cues reflecting human emotion status. While most previous approaches only extract a single speech feature for emotion recognition, existing fusion methods such as concatenation, parallel connection, and splicing ignore heterogeneous patterns in the interaction between features and features, resulting in performance of existing systems. In this paper, we propose a novel graph-based fusion method to explicitly model the relationships between every pair of speech features. Specifically, we propose a multi-dimensional edge features learning strategy called Graph-based multi-Feature fusion method for speech emotion recognition. It represents each speech feature as a node and learns multi-dimensional edge features to explicitly describe the relationship between each feature-feature pair in the context of emotion recognition. This way, the learned multi-dimensional edge features encode speech feature-level information from both the vertex and edge dimensions. Our Approach consists of three modules: an Audio Feature Generation(AFG)module, an Audio-Feature Multi-dimensional Edge Feature(AMEF) module and a Speech Emotion Recognition (SER) module. The proposed methodology yielded satisfactory outcomes on the SEWA dataset. Furthermore, the method demonstrated enhanced performance compared to the baseline in the AVEC 2019 Workshop and Challenge. We used data from two cultures as our training and validation sets: two cultures containing German and Hungarian on the SEWA dataset, the CCC scores for German are improved by 17.28% for arousal and 7.93% for liking. The outcomes of our methodology demonstrate a 13% improvement over alternative fusion techniques, including those employing one dimensional edge-based feature fusion approach.

Speech Emotion Recognition Based on a Fusion of All-Class and Pairwise-Class Feature Selection

Emotion Recognition in Videos via Fusing Multimodal Features.

Reliability detection by Fuzzy SVM with UBM Component feature for emotional speaker recognition

Speech Emotion Recognition with Emotion-Pair Based Framework Considering Emotion Distribution Information in Dimensional Emotion Space.

Speech Emotion Recognition Based on Feature Selection and Extreme Learning Machine Decision Tree

Speaker-independent Speech Emotion Recognition Based on Random Forest Feature Selection Algorithm

Speech Emotion Recognition Based on Linear Discriminant Analysis and Support Vector Machine Decision Tree

Speech emotion recognition using multimodal feature fusion with machine learning approach

Graph-based multi-Feature fusion method for speech emotion recognition

An autoencoder-based feature level fusion for speech emotion recognition

Speech Emotion Recognition Based on Multi-feature and Multi-lingual Fusion

Speech Emotion Classification with the Combination of Statistic Features and Temporal Features.

A Discriminative Feature Representation Method Based on Cascaded Attention Network With Adversarial Strategy for Speech Emotion Recognition

Combining Feature Selection And Representation For Speech Emotion Recognition

Speech Emotion Recognition Based on Syllable-Level Feature Extraction

A Feature Fusion Model with Data Augmentation for Speech Emotion Recognition

Multi-Modal Fusion Emotion Recognition Method of Speech Expression Based on Deep Learning

Feature selection for fast speech emotion recognition.

Speech emotion recognition based on multi-dimensional feature extraction and multi-scale feature fusion

Study on Feature Subspace of Archetypal Emotions for Speech Emotion Recognition

Feature Selection for Emotion Recognition of Mandarin Speech