Abstract:With the development of technology, the importance of the research on speech emotion recognition and semantic analysis has increased. The research is primarily applied in companion robot, technology products and medical purpose. In this research, a communication system with speech emotion recognition is proposed. The system pre-process speech with sound data enhancing method in speech emotion recognition and transform the sound into spectrogram by MFCC (Mel Frequency Cepstral Coefficient). Then, GoogLeNet of CNN (Convolutional Neural Network) is applied to recognize the five emotions, which are peace, happy, sad, angry and fear, and the top accuracy of recognition is 79.81%. When applying semantic analysis, the training texts are divided into two categories, positive and negative, and the chatting conversations are conducted in the framework Seq2Seq of RNN (Recurrent Neural Network). The systematic framework of this research has two parts, the client and the server. The former one is developed on Android system to be used in Application, and the latter one is established by Ubuntu Linux system and combined with the web server. With the bi-terminal framework system, the users can record voice in APP one his/her cellphone and upload the voice file to the server. Then, the voice undergoes speech emotion recognition by CNN and semantic analysis by RNN to function as a chatting machine that can respond positively or negatively based on the detected emotion and show the results on APP of the user's cell phone. The main contributions of this research are: 1) This study introduces the Chinese word vector to the robot dialogue system, effectively improving dialogue tolerance and semantic interpretation, 2) The traditional method of emotion identification must first tokenize the Chinese words, analyze the clauses and part of speech, and capture the emotional keywords before being interpreted by the expert system. Different from the traditional method, this study classifies the input directly through the convolutional neural network after the input sentence is converted into a spectrogram by MFCC, and 3) in addition to implementing the companion robot, the user's emotional index can be collected for analysis by the back-end care organization. In addition, compared with other commercial humanoid companion robots, this study is presented in an App, which is easier to use and economical.

Enhancing IVR Systems in Mobile Banking with Emotion Analysis for Adaptive Dialogue Flows and Seamless Transition to Human Assistance

Consumer Shopping Emotion and Interest Database: a Unique Database with a Multimodal Emotion Recognition Method for Retail Service Robots to Infer Consumer Shopping Intentions Better Than Humans

Speaker-Independent Speech Emotion Recognition Based On Cnn-Blstm And Multiple Svms

Enhancing speech emotion recognition through deep learning and handcrafted feature fusion

A Methodical Framework Utilizing Transforms and Biomimetic Intelligence-Based Optimization with Machine Learning for Speech Emotion Recognition

Detection of Emotion of Speech for RAVDESS Audio Using Hybrid Convolution Neural Network

Acoustic and linguistic representations for speech continuous emotion recognition in call center conversations

A CNN-Assisted Enhanced Audio Signal Processing for Speech Emotion Recognition

Detail-Enhanced Intra- and Inter-modal Interaction for Audio-Visual Emotion Recognition

A Proposal for Multimodal Emotion Recognition Using Aural Transformers and Action Units on RAVDESS Dataset

Audio-Visual Speech Enhancement in Noisy Environments via Emotion-Based Contextual Cues

A Comparison of Machine Learning Algorithms and Feature Sets for Automatic Vocal Emotion Recognition in Speech

Study on emotion recognition and companion Chatbot using deep neural network

Towards Understanding Emotions for Engaged Mental Health Conversations

Human-Computer Interaction for Recognizing Speech Emotions Using Multilayer Perceptron Classifier

Revolutionizing Mental Health Support: An Innovative Affective Mobile Framework for Dynamic, Proactive, and Context-Adaptive Conversational Agents

Emotion Inferring from Large-scale Internet Voice Data: A Multimodal Deep Learning Approach

Emotionflow: Capture the Dialogue Level Emotion Transitions

End-to-End Continuous Speech Emotion Recognition in Real-life Customer Service Call Center Conversations

Evolution of IVR building techniques: from code writing to AI-powered automation

Multimodal Emotion Recognition on RAVDESS Dataset Using Transfer Learning