Abstract:In the realm of consumer technology, Artificial Intelligence (AI)-based Speech Emotion Recognition (SER) has rapidly gained traction and integration into smart home systems. Its precision in recognition has become a pivotal factor significantly impacting user experience. However, the intricate task of selecting suitable features has emerged as a daunting challenge due to the variances in speech features induced by emotional nuances. Present research predominantly concentrates on localized speech characteristics, neglecting the broader contextual cues inherent in speech signals. This oversight contributes to relatively diminished accuracy in emotion recognition within smart home systems. To tackle this challenge, this paper introduces an enhanced Speech Emotion Recognition approach named TF-Mix. This methodology enriches emotional prediction from speech by leveraging audio data augmentation and embracing multiple features, thereby achieving superior performance in emotion recognition. To augment the model's adaptability, TF-Mix adeptly amalgamates various feature extraction techniques, encompassing Convolutional Neural Networks (CNNs), Long Short-Term Memory networks (LSTMs), and Transformer architecture. The synergy among these methodologies culminates in the formulation of three distinct architectural models. The primary architecture is founded on a 1-dimensional Convolutional Neural Network (CNN), closely followed by a Fully Connected Network (FCN). Subsequent architectures, notably BiLSTM-FCN and BiLSTM-Transformer-FCN, retain their respective structures while incorporating CNNs. Moreover, the amalgamation of individual models into an ensemble model, designated as D, via weighted averaging, further amplifies the efficacy of emotion recognition. Experimental outcomes showcase exceptional performance across all four models in the SER task. The ensemble Model D achieves noteworthy accuracy across multiple datasets: 87.513% on RAVDESS, 86.233% on SAVEE, 99.857% on TESS, 82.295% on CREMA-D, and 97.546% on the TOTAL dataset.

A Multi-Feature Multi-Classifier System for Speech Emotion Recognition

Emotion Recognition in Videos via Fusing Multimodal Features.

Speech Emotion Recognition Using Multiple Classifiers

Speech Emotion Recognition Based on Feature Selection and Extreme Learning Machine Decision Tree

Visual-Audio Emotion Recognition Based on Multi-Task and Ensemble Learning with Multiple Features

Speech Emotion Recognition Based on Linear Discriminant Analysis and Support Vector Machine Decision Tree

Speech Emotion Recognition Based on Formant Characteristics Feature Extraction and Phoneme Type Convergence.

Multi-Classifier Interactive Learning for Ambiguous Speech Emotion Recognition

Speech emotion recognition using a novel feature set

Speech Emotion Classification Using Acoustic Features

Multi-feature Fusion Speech Emotion Recognition Based on SVM

Smart Multi-Model Emotion Recognition System with Deep learning

Paralinguistic and spectral feature extraction for speech emotion classification using machine learning techniques

Multi-modal Emotion Recognition Based on Speech and Image.

Speech emotion recognition using multimodal feature fusion with machine learning approach

Emotion recognition using support vector machine and deep neural network

Comprehensive Speech Emotion Recognition System Employing Multi-Layer Perceptron (MLP) Classifier and libRosa Feature Extraction

Design of smart home system speech emotion recognition model based on ensemble deep learning and feature fusion

Speech emotion recognition based on multi-dimensional feature extraction and multi-scale feature fusion

Learning multi-scale features for speech emotion recognition with connection attention mechanism

Multi-stage Classification of Emotional Speech Motivated by a Dimensional Emotion Model