Abstract:In the realm of consumer technology, Artificial Intelligence (AI)-based Speech Emotion Recognition (SER) has rapidly gained traction and integration into smart home systems. Its precision in recognition has become a pivotal factor significantly impacting user experience. However, the intricate task of selecting suitable features has emerged as a daunting challenge due to the variances in speech features induced by emotional nuances. Present research predominantly concentrates on localized speech characteristics, neglecting the broader contextual cues inherent in speech signals. This oversight contributes to relatively diminished accuracy in emotion recognition within smart home systems. To tackle this challenge, this paper introduces an enhanced Speech Emotion Recognition approach named TF-Mix. This methodology enriches emotional prediction from speech by leveraging audio data augmentation and embracing multiple features, thereby achieving superior performance in emotion recognition. To augment the model's adaptability, TF-Mix adeptly amalgamates various feature extraction techniques, encompassing Convolutional Neural Networks (CNNs), Long Short-Term Memory networks (LSTMs), and Transformer architecture. The synergy among these methodologies culminates in the formulation of three distinct architectural models. The primary architecture is founded on a 1-dimensional Convolutional Neural Network (CNN), closely followed by a Fully Connected Network (FCN). Subsequent architectures, notably BiLSTM-FCN and BiLSTM-Transformer-FCN, retain their respective structures while incorporating CNNs. Moreover, the amalgamation of individual models into an ensemble model, designated as D, via weighted averaging, further amplifies the efficacy of emotion recognition. Experimental outcomes showcase exceptional performance across all four models in the SER task. The ensemble Model D achieves noteworthy accuracy across multiple datasets: 87.513% on RAVDESS, 86.233% on SAVEE, 99.857% on TESS, 82.295% on CREMA-D, and 97.546% on the TOTAL dataset.

Iself: Towards Cold-Start Emotion Labeling Using Transfer Learning With Smartphones

iSelf: Towards cold-start emotion labeling using transfer learning with smartphones

Self-attention Transfer Networks for Speech Emotion Recognition

Better Than Humans: a Method for Inferring Consumer Shopping Intentions by Reading Facial Expressions

Consumer Shopping Emotion and Interest Database: a Unique Database with a Multimodal Emotion Recognition Method for Retail Service Robots to Infer Consumer Shopping Intentions Better Than Humans

Inferring Mood Instability Via Smartphone Sensing

A Social-Driven Intelligent System to Assist the Classification of Pet Emotions using Deep Learning and Big Data Analysis

MoodExplorer

A Hybrid Emotion Recognition on Android Smart Phones.

PersonalitySensing: A Multi-View Multi-Task Learning Approach for Personality Detection Based on Smartphone Usage

Fast Yet Effective Speech Emotion Recognition with Self-distillation

EmotionTracker: A Mobile Real-time Facial Expression Tracking System with the Assistant of Public AI-as-a-Service

Personalized Broad Learning System for Facial Expression

Autonomous learning of foreign language based on facial emotion recognition and cloud computing

Personalized Emotion Recognition and Emotion Prediction System Based on Cloud Computing

Design of smart home system speech emotion recognition model based on ensemble deep learning and feature fusion

Self-labeling with feature transfer for speech emotion recognition

Behavioral and Physiological Signals-Based Deep Multimodal Approach for Mobile Emotion Recognition

Demo: LIVES: Learning through Interactive Video and Emotion-aware System.

Self-Labeling Learning Ensemble via Deep Recurrent Neural Network and Self-Representation for Speech Emotion Recognition

Learning to Infer Public Emotions from Large-Scale Networked Voice Data