Abstract:An ensemble visual-audio emotion recognition framework is proposed based on multi-task and blending learning with multiple features in this paper. To solve the problem that existing features can not accurately identify different emotions, we extract two kinds features, i. e., Interspeech 2010 and deep features for audio data, LBP and deep features for visual data, with the intent to accurately identify different emotions by using different features. Owing to the diversity of features, SVM classifiers and CNN are designed for manual features, i.e., Interspeech 2010 features and local LBP features, and deep features, through which four sub-models are obtained. Finally, the blending ensemble algorithm is used to fuse sub-models to improve the recognition performance of visual-audio emotion recognition. In addition, multi-task learning is applied in the CNN model for deep features, which can predict multiple tasks at the same time with fewer parameters and improve the sensitivity of the single recognition model to user’s emotion by sharing information between different tasks. Experiments are performed using eNTERFACCE database, from which the results indicate that the recognition of multi-task CNN increased by 3% and 2% on average over CNN model in speaker-independent and speaker-dependent experiments, respectively. And emotion recognition accuracy of visual-audio by our method reaches 81.36% and 78.42% in speaker-independent and speaker-dependent experiments, respectively, which maintain higher performance than some state-of-the-art works.

Speaker Recognition Algorithm Based on Neural Network Ensemble and Its Simulation Study

Emotional speaker recognition based on similar neighbor phenomenon

Speaker Recognition Based on Long Short-Term Memory Networks

Method for training speaker recognition network model, speaker recognition method and system

Self-attention Based Speaker Recognition Using Cluster-Range Loss

Visual-Audio Emotion Recognition Based on Multi-Task and Ensemble Learning with Multiple Features

Speaker Identification Using Wavelet Shannon Entropy and Probabilistic Neural Network

Parametric feature-based voice recognition system using artificial neural network

Deep neural networks based speaker modeling at different levels of phonetic granularity

Hybrid Architecture Based on Fuzzy Classifier and Multiplayer Feed-Forward Neural Network for Speaker Identification

Speech Recognition Algorithm Based on Neural Network and Hidden Markov Model

Text-independent Speaker Recognition Based on X-vector

Speaker Recognition on Mobile Phone: Using Wavelet, Cepstral Coefficients and Probabilisitc Neural Network

A novel speech feature fusion algorithm for text-independent speaker recognition

Neural Scoring, Not Embedding: A Novel Framework for Robust Speaker Verification

Speaker Recognition System Based on Deep Neural Networks and Bottleneck Features

DSARSR: Deep Stacked Auto-encoders Enhanced Robust Speaker Recognition

Auditory Model Based Speech Feature Extraction and Its Application to Speaker Identification

Speaker Identification System Based on Hybrid Neural Network

Speech Emotion Recognition Based on Convolutional Neural Network and Feature Fusion

An Anti-noise Algorithm for Speaker-Dependent Mandarin Vocal Numbers Recognition