Abstract:In the realm of consumer technology, Artificial Intelligence (AI)-based Speech Emotion Recognition (SER) has rapidly gained traction and integration into smart home systems. Its precision in recognition has become a pivotal factor significantly impacting user experience. However, the intricate task of selecting suitable features has emerged as a daunting challenge due to the variances in speech features induced by emotional nuances. Present research predominantly concentrates on localized speech characteristics, neglecting the broader contextual cues inherent in speech signals. This oversight contributes to relatively diminished accuracy in emotion recognition within smart home systems. To tackle this challenge, this paper introduces an enhanced Speech Emotion Recognition approach named TF-Mix. This methodology enriches emotional prediction from speech by leveraging audio data augmentation and embracing multiple features, thereby achieving superior performance in emotion recognition. To augment the model's adaptability, TF-Mix adeptly amalgamates various feature extraction techniques, encompassing Convolutional Neural Networks (CNNs), Long Short-Term Memory networks (LSTMs), and Transformer architecture. The synergy among these methodologies culminates in the formulation of three distinct architectural models. The primary architecture is founded on a 1-dimensional Convolutional Neural Network (CNN), closely followed by a Fully Connected Network (FCN). Subsequent architectures, notably BiLSTM-FCN and BiLSTM-Transformer-FCN, retain their respective structures while incorporating CNNs. Moreover, the amalgamation of individual models into an ensemble model, designated as D, via weighted averaging, further amplifies the efficacy of emotion recognition. Experimental outcomes showcase exceptional performance across all four models in the SER task. The ensemble Model D achieves noteworthy accuracy across multiple datasets: 87.513% on RAVDESS, 86.233% on SAVEE, 99.857% on TESS, 82.295% on CREMA-D, and 97.546% on the TOTAL dataset.

Driver’s Speech Emotion Recognition for Smart Cockpit Based on a Self-Attention Deep Learning Framework

Self-attention Transfer Networks for Speech Emotion Recognition

Global-Local-Feature-Fused Driver Speech Emotion Detection for Intelligent Cockpit in Automated Driving.

Speech Emotion Recognition Based on Convolutional Neural Network with Attention-Based Bidirectional Long Short-Term Memory Network and Multi-Task Learning

·Ai-Enabled Intelligent Cockpit Proactive Affective Interaction: Middle-Level Feature Fusion Dual-Branch Deep Learning Network for Driver Emotion Recognition

Design of smart home system speech emotion recognition model based on ensemble deep learning and feature fusion

Speaker-Independent Speech Emotion Recognition Based On Cnn-Blstm And Multiple Svms

Drivers' Comprehensive Emotion Recognition Based on HAM

Speech Emotion Recognition Using Multi-Modal Feature Fusion Network

Intelligent In-Car Emotion Regulation Interaction System Based on Speech Emotion Recognition

A Feature Fusion Model with Data Augmentation for Speech Emotion Recognition

Visual-Audio Emotion Recognition Based on Multi-Task and Ensemble Learning with Multiple Features

Learning multi-scale features for speech emotion recognition with connection attention mechanism

Speech Emotion Recognition by Combining a Unified First-Order Attention Network with Data Balance

Dilated Residual Network with Multi-head Self-attention for Speech Emotion Recognition

Driver Emotion Recognition with a Hybrid Attentional Multimodal Fusion Framework

Multi-type Features Separating Fusion Learning for Speech Emotion Recognition.

Multimodal Speech Emotion Recognition Based on Multi-Scale MFCCs and Multi-View Attention Mechanism

Research on Deep Learning-based Speech Emotion Recognition System

Ms-senet: Enhancing Speech Emotion Recognition Through Multi-scale Feature Fusion With Squeeze-and-excitation Blocks

CogEmoNet: A Cognitive-Feature-Augmented Driver Emotion Recognition Model for Smart Cockpit