Abstract:In recent years, the increasing popularity of smart mobile devices has made the interaction between devices and users, particularly through voice interaction, more crucial. By enabling smart devices to better understand users' emotional states through voice data, it becomes possible to provide more personalized services. This paper proposes a novel machine learning model for speech emotion recognition called CLDNN, which combines convolutional neural networks (CNN), long short-term memory neural networks (LSTM), and deep neural networks (DNN). To design a system that closely resembles the human auditory system in recognizing audio signals, this article uses the Mel-frequency cepstral coefficients (MFCCs) of audio data as the input of the machine learning model. First, the MFCCs of the voice signal are extracted as the input of the model. Local feature learning blocks (LFLBs) composed of one-dimensional CNNs are employed to calculate the feature values of the data. As audio signals are time-series data, the resulting feature values from LFLBs are then fed into the LSTM layer to enhance learning on the time-series level. Finally, fully connected layers are used for classification and prediction. The experimental evaluation of the proposed model utilizes three databases: RAVDESS, EMO-DB, and IEMOCAP. The results demonstrate that the LSTM model effectively models the features extracted from the 1D CNN due to the time-series characteristics of speech signals. Additionally, the data augmentation method applied in this paper proves beneficial in improving the recognition accuracy and stability of the systems for different databases. Furthermore, according to the experimental results, the proposed system achieves superior recognition rates compared to related research in speech emotion recognition.

Improve Data Utilization with Two-stage Learning in CNN-LSTM-based Voice Activity Detection

A Novel and Efficient Voice Activity Detector Using Shape Features of Speech Wave.

Multi-task Joint-Learning for Robust Voice Activity Detection

Performance Improvement of Speech Emotion Recognition Systems by Combining 1D CNN and LSTM with Data Augmentation

Voice Activity Detection Based on Time-Delay Neural Networks

Voice activity detection in the wild: A data-driven approach using teacher-student training

Boosting Contextual Information for Deep Neural Network Based Voice Activity Detection

A Universal VAD Based on Jointly Trained Deep Neural Networks.

Bimodal Recurrent Neural Network for Audiovisual Voice Activity Detection

DNN-based Voice Activity Detection for Speaker Recognition

Denoising Deep Neural Networks Based Voice Activity Detection

Voice activity detection using a local-global attention model

Boosted deep neural networks and multi-resolution cochleagram features for voice activity detection.

Incorporating VAD into ASR System by Multi-task Learning

Speech Enhancement Method Based on LSTM Neural Network for Speech Recognition

End-to-End Speaker-Dependent Voice Activity Detection

AUC Optimization for Deep Learning Based Voice Activity Detection.

Speech enhancement aided end-to-end multi-task learning for voice activity detection

Multimodal Voice Activity Detection

A Comparative Study of Robustness of Deep Learning Approaches for VAD

Deep Belief Networks Based Voice Activity Detection