Abstract:In the realm of consumer technology, Artificial Intelligence (AI)-based Speech Emotion Recognition (SER) has rapidly gained traction and integration into smart home systems. Its precision in recognition has become a pivotal factor significantly impacting user experience. However, the intricate task of selecting suitable features has emerged as a daunting challenge due to the variances in speech features induced by emotional nuances. Present research predominantly concentrates on localized speech characteristics, neglecting the broader contextual cues inherent in speech signals. This oversight contributes to relatively diminished accuracy in emotion recognition within smart home systems. To tackle this challenge, this paper introduces an enhanced Speech Emotion Recognition approach named TF-Mix. This methodology enriches emotional prediction from speech by leveraging audio data augmentation and embracing multiple features, thereby achieving superior performance in emotion recognition. To augment the model's adaptability, TF-Mix adeptly amalgamates various feature extraction techniques, encompassing Convolutional Neural Networks (CNNs), Long Short-Term Memory networks (LSTMs), and Transformer architecture. The synergy among these methodologies culminates in the formulation of three distinct architectural models. The primary architecture is founded on a 1-dimensional Convolutional Neural Network (CNN), closely followed by a Fully Connected Network (FCN). Subsequent architectures, notably BiLSTM-FCN and BiLSTM-Transformer-FCN, retain their respective structures while incorporating CNNs. Moreover, the amalgamation of individual models into an ensemble model, designated as D, via weighted averaging, further amplifies the efficacy of emotion recognition. Experimental outcomes showcase exceptional performance across all four models in the SER task. The ensemble Model D achieves noteworthy accuracy across multiple datasets: 87.513% on RAVDESS, 86.233% on SAVEE, 99.857% on TESS, 82.295% on CREMA-D, and 97.546% on the TOTAL dataset.

Speech Emotion Recognition under Resource Constraints with Data Distillation

Fast Yet Effective Speech Emotion Recognition with Self-distillation

Self-attention Transfer Networks for Speech Emotion Recognition

Speech Emotion Recognition Based on Formant Characteristics Feature Extraction and Phoneme Type Convergence.

Speech Emotion Recognition Based on Clustering Assistance

Knowledge Transfer For On-Device Speech Emotion Recognition with Neural Structured Learning

hierarchical network with decoupled knowledge distillation for speech emotion recognition

Electroglottograph-Based Speech Emotion Recognition Via Cross-Modal Distillation

Speech Emotion Recognition with Distilled Prosodic and Linguistic Affect Representations

Design of smart home system speech emotion recognition model based on ensemble deep learning and feature fusion

Leveraging Semantic Information for Efficient Self-Supervised Emotion Recognition with Audio-Textual Distilled Models

TrustSER: On the Trustworthiness of Fine-tuning Pre-trained Speech Embeddings For Speech Emotion Recognition

An HASM-Assisted Voice Disguise Scheme for Emotion Recognition of IoT-enabled Voice Interface

Speech Emotion Recognition Via Multi-Level Cross-Modal Distillation

A Preliminary Study on Augmenting Speech Emotion Recognition using a Diffusion Model

Strong Generalized Speech Emotion Recognition Based on Effective Data Augmentation

Speaker to Emotion: Domain Adaptation for Speech Emotion Recognition with Residual Adapters.

Speech Emotion Recognition Exploiting ASR-based and Phonological Knowledge Representations.

A Breakthrough in Speech Emotion Recognition Using Deep Retinal Convolution Neural Networks.

Multi-Level Knowledge Distillation for Speech Emotion Recognition in Noisy Conditions

Dilated Residual Network with Multi-head Self-attention for Speech Emotion Recognition