Emotion Recognition via Continuous Mandarin Speech

Tsang-Long Pao,Jun-Heng Yeh,Yu-Te Che

DOI: https://doi.org/10.5772/5921

2008-10-01

Abstract:Emotion Recognition via Continuous Mandarin Speech | InTechOpen, Published on: 2008-10-01. Authors: Tsang-Long Pao, Jun-Heng Yeh and Yu-Te Chen

What problem does this paper attempt to address?

Exploring Spatio-Temporal Representations by Integrating Attention-based Bidirectional-LSTM-RNNs and FCNs for Speech Emotion Recognition

Ziping Zhao,Yu Zheng,Zixing Zhang,Haishuai Wang,Yiqin Zhao,Chao Li

DOI: https://doi.org/10.21437/interspeech.2018-1477

2018-01-01

Abstract:Automatic emotion recognition from speech, which is an important and challenging task in the field of affective computing, heavily relies on the effectiveness of the speech features for classification. Previous approaches to emotion recognition have mostly focused on the extraction of carefully hand-crafted features. How to model spatio-temporal dynamics for speech emotion recognition effectively is still under active investigation. In this paper, we propose a method to tackle the problem of emotional relevant feature extraction from speech by leveraging Attention-based Bidirectional Long Short-Term Memory Recurrent Neural Networks with fully convolutional networks in order to automatically learn the best spatio-temporal representations of speech signals. The learned high-level features are then fed into a deep neural network (DNN) to predict the final emotion. The experimental results on the Chinese Natural Audio-Visual Emotion Database (CHEAVD) and the Interactive Emotional Dyadic Motion Capture (IEMOCAP) corpora show that our method provides more accurate predictions compared with other existing emotion recognition algorithms.
EmoEars: an emotion recognition system for mandarin speech

Bo Xie,Ling Chen,Gen-Cai Chen,Chun Chen

DOI: https://doi.org/10.1007/11596448_140

2005-01-01

Abstract:In this paper, an emotion recognition system for mandarin speech is presented. Five basic human emotions including angry, fear, happy, neutral and sad are investigated. The recognizer is based on neural network with OCON and ACON architecture. Some novel feature selection methods are also added as optional tool to enhance the efficiency and classification accuracy. The system can train speaker dependent emotion speech model through online emotional utterance recording. Experiment results show that emotion can be recognized through neural network model, the best mean accuracy is 86.7%. In addition, the feature selection module is effective to reduce the compute load and increase the generalization ability of the recognizer.
Cost-Sensitive Learning for Emotion Robust Speaker Recognition

Dongdong Li,Yingchun Yang,Weihui Dai

DOI: https://doi.org/10.1155/2014/628516

2014-01-01

Abstract:In the field of information security, voice is one of the most important parts in biometrics. Especially, with the development of voice communication through the Internet or telephone system, huge voice data resources are accessed. In speaker recognition, voiceprint can be applied as the unique password for the user to prove his/her identity. However, speech with various emotions can cause an unacceptably high error rate and aggravate the performance of speaker recognition system. This paper deals with this problem by introducing a cost-sensitive learning technology to reweight the probability of test affective utterances in the pitch envelop level, which can enhance the robustness in emotion-dependent speaker recognition effectively. Based on that technology, a new architecture of recognition system as well as its components is proposed in this paper. The experiment conducted on the Mandarin Affective Speech Corpus shows that an improvement of 8% identification rate over the traditional speaker recognition is achieved.
Mandarin Emotion Recognition Combining Acoustic and Emotional Point Information

Lijiang Chen,Xia Mao,Pengfei Wei,Yuli Xue,Mitsuru Ishizuka

DOI: https://doi.org/10.1007/s10489-012-0352-1

IF: 5.3

2012-01-01

Applied Intelligence

Abstract:In this contribution, we introduce a novel approach to combine acoustic information and emotional point information for a robust automatic recognition of a speaker’s emotion. Six discrete emotional states are recognized in the work. Firstly, a multi-level model for emotion recognition by acoustic features is presented. The derived features are selected by fisher rate to distinguish different types of emotions. Secondly, a novel emotional point model for Mandarin is established by Support Vector Machine and Hidden Markov Model. This model contains 28 emotional syllables which reflect rich emotional information. Finally the acoustic information and emotional point information are integrated by a soft decision strategy. Experimental results show that the application of emotional point information in speech emotion recognition is effective.
Emotion Recognition and Conversion for Mandarin Speech

Yu Zhou,Jianping Zhang,Ling Wang,Yonghong Yan

DOI: https://doi.org/10.1109/fskd.2009.474

2009-01-01

Abstract:In this study, some research activities on expressive speech recognition and conversion will be introduced. A database consisting of five kinds of speech emotions (i.e. happiness, sadness, surprise, anger and neutral) is used. Not only those traditional features such as mfcc, plp, and pitch are studied, but also a new feature extraction method based on fisher's F-Ratio is proposed and reported. In our experiments, various combinations of these features, including their high order features are applied using GMM modeling for Mandarin expressive speech recognition. Also we presented some results from emotional speech conversion with a pitch target model.
Real-time Speech Emotion Recognition Based on Syllable-Level Feature Extraction

Abdul Rehman,Zhentao Liu,Min Wang,Wenming Cao,Cheng-Shan Jiang

DOI: https://doi.org/10.48550/arxiv.2204.11382

2022-01-01

Abstract:Speech emotion recognition systems have high prediction latency because of the high computational requirements for deep learning models and low generalizability mainly because of the poor reliability of emotional measurements across multiple corpora. To solve these problems, we present a speech emotion recognition system based on a reductionist approach of decomposing and analyzing syllable-level features. Mel-spectrogram of an audio stream is decomposed into syllable-level components, which are then analyzed to extract statistical features. The proposed method uses formant attention, noise-gate filtering, and rolling normalization contexts to increase feature processing speed and tolerance to adversity. A set of syllable-level formant features is extracted and fed into a single hidden layer neural network that makes predictions for each syllable as opposed to the conventional approach of using a sophisticated deep learner to make sentence-wide predictions. The syllable level predictions help to achieve the real-time latency and lower the aggregated error in utterance level cross-corpus predictions. The experiments on IEMOCAP (IE), MSP-Improv (MI), and RAVDESS (RA) databases show that the method archives real-time latency while predicting with state-of-the-art cross-corpus unweighted accuracy of 47.6% for IE to MI and 56.2% for MI to IE.
Emotion Recognition From Noisy Speech

Mingyu You,Chun Chen,Jiajun Bu,Jia Liu,Jianhua Tao

DOI: https://doi.org/10.1109/ICME.2006.262865

2006-01-01

Abstract:This paper presents an emotion recognition system from clean and noisy speech. Geodesic distance was adopted to preserve the intrinsic geometry of emotional speech. Based on the geodesic distance estimation, an enhanced Lipschitz embedding was developed to embed the 64-dimensional acoustic features into a six-dimensional space. In order to avoid the problems brought by noise reduction, emotion recognition from noisy speech was performed directly. Linear Discriminant Analysis (LDA), Principal Component Analysis (PCA) and feature selection by Sequential Forward Selection (SFS) with Support Vector Machine (SVM) were also included to compress acoustic features before classifying the emotional states of clean and noisy speech. Experimental results demonstrate that compared with other methods, the proposed system makes approximately 10% improvement. The performance of our system is also robust when speech data is corrupted by increasing noise.
Semi-Automatic Emotion Recognition From Chinese Text

Yu Zhang,Fuji Ren,Shingo Kuroiwa

2006-01-01

Abstract:The interest in emotion recognition has attracted more and more attentions. There have been some studies about emotion extraction from spoken natural language dialog, and most of them have successfully been developed within the specified domains. But computer application on Chinese Natural Language Processing (NLP) is still on the starting stage. Aiming at developing a Chinese affective dialogue system, we have been exploring how to incorporate emotional aspects of dialogue into existing dialogue processing techniques. However, in Chinese linguistic corpora only few of them include emotional information. As a preliminary step toward this goal, we work on constructing what we call an emotion-thesaurus, as the base of our emotion recognition system. Then we construct the classification system using the thesaurus. Finally we give the evaluation of our system.
Speech Emotion Recognition Based on Syllable-Level Feature Extraction

Abdul Rehman Javed,Zhentao Liu,Min Wu,Weihua Cao,Cheng-Shan Jiang

DOI: https://doi.org/10.2139/ssrn.4372645

2023-01-01

Abstract:Download This Paper Open PDF in Browser Add Paper to My Library Share: Permalink Using these links will ensure access to this page indefinitely Copy URL Copy DOI
Speech Emotion Recognition Based on Formant Characteristics Feature Extraction and Phoneme Type Convergence.

Zhen-Tao Liu,Abdul Rehman,Min Wu,Wei-Hua Cao,Man Hao

DOI: https://doi.org/10.1016/j.ins.2021.02.016

IF: 8.1

2021-01-01

Information Sciences

Abstract:Speech Emotion Recognition (SER) has numerous applications including human-robot interaction, online gaming, and health care assistance. While deep learning-based approaches achieve considerable precision, they often come with high computational and time costs. Indeed, feature learning strategies must search for important features in a large amount of speech data. In order to reduce these time and computational costs, we propose pre-processing step in which speech segments with similar formant characteristics are clustered together and labeled as the same phoneme. The phoneme occurrence rates in emotional utterances are then used as the input features for classifiers. Using six databases (EmoDB, RAVDESS, IEMOCAP, ShEMO, DEMoS and MSP-Improv) for evaluation, the level of accuracy is comparable to that of current state-of-the-art methods and the required training time was significantly reduced from hours to minutes. (c) 2021 Elsevier Inc. All rights reserved.
Emotion Recognition From Speech and Text using Long Short-Term Memory

Sonagiri China Venkateswarlu,Siva Ramakrishna Jeevakala,Naluguru Udaya Kumar,Pidugu Munaswamy,Dhanalaxmi Pendyala

DOI: https://doi.org/10.48084/etasr.6004

2023-08-09

Abstract:Everyday interactions depend on more than just rational discourse; they also depend on emotional reactions. Having this information is crucial to making any kind of practical or even rational decision, as it can help to better understand one another by sharing our responses and providing recommendations on how they may feel. Several studies have recently begun to focus on emotion detection and labeling, proposing different methods for organizing feelings and detecting emotions in speech. Determining how emotions are conveyed through speech has been given major emphasis in social interactions during the last decade. However, the real efficiency of identification needs to be improved because of the severe lack of data on the primary temporal link of the speech waveform. Currently, a new approach to speech recognition is recommended, which couples structured audio information with long-term neural networks to fully take advantage of the shift in emotional content across phases. In addition to time series characteristics, structural speech features taken from the waveforms are now in charge of maintaining the underlying connection between layers of the actual speech. There are several Long-Short-Term Memory (LSTM) based algorithms for identifying emotional focus over numerous blocks. The proposed method (i) reduced overhead by optimizing the standard forgetting gate, reducing the amount of required processing time, (ii) applied an attention mechanism to both the time and feature dimension in the LSTM's final output to get task-related information, rather than using the output from the prior iteration of the standard technique, and (iii) employed a powerful strategy to locate the spatial characteristics in the final output of the LSTM to gain information, as opposed to using the findings from the prior phase of the regular method. The proposed method achieved an overall classification accuracy of 96.81%.
Research on Chinese Speech Emotion Recognition Based on Deep Neural Network and Acoustic Features

Ming-Che Lee,Sheng-Cheng Yeh,Jia-Wei Chang,Zhen-Yi Chen

DOI: https://doi.org/10.3390/s22134744

2022-06-23

Abstract:In recent years, the use of Artificial Intelligence for emotion recognition has attracted much attention. The industrial applicability of emotion recognition is quite comprehensive and has good development potential. This research uses voice emotion recognition technology to apply it to Chinese speech emotion recognition. The main purpose of this research is to transform gradually popularized smart home voice assistants or AI system service robots from a touch-sensitive interface to a voice operation. This research proposed a specifically designed Deep Neural Network (DNN) model to develop a Chinese speech emotion recognition system. In this research, 29 acoustic characteristics in acoustic theory are used as the training attributes of the proposed model. This research also proposes a variety of audio adjustment methods to amplify datasets and enhance training accuracy, including waveform adjustment, pitch adjustment, and pre-emphasize. This study achieved an average emotion recognition accuracy of 88.9% in the CASIA Chinese sentiment corpus. The results show that the deep learning model and audio adjustment method proposed in this study can effectively identify the emotions of Chinese short sentences and can be applied to Chinese voice assistants or integrated with other dialogue applications.
Manifolds Based Emotion Recognition in Speech.

Mingyu You,Chun Chen,Jiajun Bu,Jia Liu,Jianhua Tao

2007-01-01

Abstract:The paper presents an emotional speech recognition system with the analysis of manifolds of speech. Working with large volumes of high-dimensio nal acoustic features, the researchers confront the problem of dimensionality reduction. Unlike classical techniques, such as Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA), a new approach, named Enhanced Lipschitz Embedding (ELE) is proposed in the paper to discover the nonlinear degrees of freedom that underlie the emotional speech corpus. ELE adopts geodesic distance to preserve the intrinsic geometry at all scales of speech corpus. Based on geodesic distance estimation, ELE embeds the 64-dimensional acoustic features into a six-dimensional space in which speech data with the same emotional state are generally clustered around one plane and the data distribution feature is beneficial to emotion classification. The compressed testing data is classified into six emotional states (neutral, anger, fear, happiness, sadness and surprise) by a trained linear Support Vector Machine (SVM) system. Considering the perception constancy of humans, ELE is also investigated in terms of its ability to detect the intrinsic geometry of emotional speech corrupted by noise. The performance of the new approach is compared with the methods of feature selection by Sequential Forward Selection (SFS), PCA, LDA, Isomap and Locally Linear Embedding (LLE). Experimental results demonstrate that, compared with other methods, the proposed system gives 9%-26% relative improvement in speaker-indep endent emotion recognition and 5%-20% improvement in speaker-dependent recognition. Meanwhile, the proposed system shows robustness and an improvement of approximately 10% in emotion recognition accuracy when speech is corrupted by increasing noise.
MES-P: an Emotional Tonal Speech Dataset in Mandarin Chinese with Distal and Proximal Labels

Zhongzhe Xiao,Ying Chen,Weibei Dou,Zhi Tao,Liming Chen

DOI: https://doi.org/10.48550/arXiv.1808.10095

2018-08-30

Sound

Abstract:Emotion shapes all aspects of our interpersonal and intellectual experiences. Its automatic analysis has there-fore many applications, e.g., human-machine interface. In this paper, we propose an emotional tonal speech dataset, namely Mandarin Chinese Emotional Speech Dataset - Portrayed (MES-P), with both distal and proximal labels. In contrast with state of the art emotional speech datasets which are only focused on perceived emotions, the proposed MES-P dataset includes not only perceived emotions with their proximal labels but also intended emotions with distal labels, thereby making it possible to study human emotional intelligence, i.e. people emotion expression ability and their skill of understanding emotions, thus explicitly accounting for perception differences between intended and perceived emotions in speech signals and enabling studies of emotional misunderstandings which often occur in real life. Furthermore, the proposed MES-P dataset also captures a main feature of tonal languages, i.e., tonal variations, and provides recorded emotional speech samples whose tonal variations match the tonal distribution in real life Mandarin Chinese. Besides, the proposed MES-P dataset features emotion intensity variations as well, and includes both moderate and intense versions of recordings for joy, anger, and sadness in addition to neutral speech. Ratings of the collected speech samples are made in valence-arousal space through continuous coordinate locations, resulting in an emotional distribution pattern in 2D VA space. The consistency between the speakers' emotional intentions and the listeners' perceptions is also studied using Cohen's Kappa coefficients. Finally, we also carry out extensive experiments using a baseline on MES-P for automatic emotion recognition and compare the results with human emotion intelligence.
English speech emotion recognition method based on speech recognition

Man Liu

DOI: https://doi.org/10.1007/s10772-021-09955-4

2022-02-08

International Journal of Speech Technology

Abstract:Speech emotion reflects important information other than text content in speech signal, while traditional speech recognition often ignores the emotion of text content, so it is difficult to understand more abundant emotional content from English text. In order to change this situation and get more emotional information from English texts, it is necessary to understand English speech emotion recognition. However, at present, the research on speech emotion recognition technology in China mainly focuses on Chinese, while the research on English speech emotion recognition is relatively few. Therefore, this paper studies English speech emotion recognition. The digital processing of speech signal is based on speech recognition. The digitization of speech signal is the premise of computer processing and analysis of speech signal. The preprocessing of speech signal can also be called front-end processing. The specific steps are: sampling and quantization, pre intensity and windowing. Voice endpoint detection is based on high-order differentiation of volume and waveform. In feature extraction, open smile is selected as the tool to directly extract features, libsvm is selected to establish speech emotion recognition model, and finally an experimental environment is built to verify the design method. The experimental results show that this method can better recognize the emotion of English speech and realize a high degree of human–computer interaction.
Study on Emotion Feature Analysis and Recognition in Speech Signal: an Overview

YU Ling-li,CAI Zi-xing,CHEN Ming-yi

DOI: https://doi.org/10.3969/j.issn.1007-0249.2007.04.016

2007-01-01

Abstract:Emotion recognition of speech is becoming one of new research projects. This paper introduces the status of emotion recognition in speech signal at home and abroad recently. And it also exposes various human emotion classifications. The parameters extraction methods and the meaning of different feature parameters are induced. Based on these, the research progress and the advantages and disadvantages of some capital constitutive modeling methods are summarized. At last, the future challenges and development trends of emotion recognition of speech are presented.
Mandarin Emotion Recognition Based on Multifractal Theory Towards Human-Robot Interaction

Hong Liu,Wenjuan Zhang

DOI: https://doi.org/10.1109/robio.2013.6739524

2013-01-01

Abstract:Emotion recognition is crucially related with friendly and humanistic human-robot interaction. Our paper aims at developing a new kind of features to mandarin emotional speech signal based on multifractal theory. Firstly, phase space structure differentiate with respect of initials and finals indicate the fractal phenomenon during speech produce process. Further, positive largest Lyapunov exponent proved existing chaos. To quantitatively measure the chaos, extension of fractal concept-multifractal is calculated by multi-fractal detrended fluctuation analysis (MFDFA) and Legendre transformation. Besides, the underlying fractal characteristics during calculation process is analyzed, which further verifies the emotional speech is multifractal rather than monofractal. Multifractal spectrum visually shows that various emotion is differentiated with each other. After extracting parameters of mulfractal spectrum, several comparative experiments are established, which is implemented with BP neural network and support vector machine (SVM) respectively to hence the comparison between our approach and conventional one. At last, improvement in recognition accuracies demonstrates that our method is available and effective.
A Combined Cepstral Distance Method for Emotional Speech Recognition

Changqin Quan,Bin Zhang,Xiao Sun,Fuji Ren

DOI: https://doi.org/10.1177/1729881417719836

IF: 1.714

2017-01-01

International Journal of Advanced Robotic Systems

Abstract:Affective computing is not only the direction of reform in artificial intelligence but also exemplification of the advanced intelligent machines. Emotion is the biggest difference between human and machine. If the machine behaves with emotion, then the machine will be accepted by more people. Voice is the most natural and can be easily understood and accepted manner in daily communication. The recognition of emotional voice is an important field of artificial intelligence. However, in recognition of emotions, there often exists the phenomenon that two emotions are particularly vulnerable to confusion. This article presents a combined cepstral distance method in two-group multi-class emotion classification for emotional speech recognition. Cepstral distance combined with speech energy is well used as speech signal endpoint detection in speech recognition. In this work, the use of cepstral distance aims to measure the similarity between frames in emotional signals and in neutral signals. These features are input for directed acyclic graph support vector machine classification. Finally, a two-group classification strategy is adopted to solve confusion in multi-emotion recognition. In the experiments, Chinese mandarin emotion database is used and a large training set (1134 + 378 utterances) ensures a powerful modelling capability for predicting emotion. The experimental results show that cepstral distance increases the recognition rate of emotion sad and can balance the recognition results with eliminating the over fitting. And for the German corpus Berlin emotional speech database, the recognition rate between sad and boring, which are very difficult to distinguish, is up to 95.45%.
Attention Based Fully Convolutional Network for Speech Emotion Recognition

Yuanyuan Zhang,Jun Du,Zirui Wang,Jianshu Zhang

DOI: https://doi.org/10.23919/APSIPA.2018.8659587

2019-05-02

Abstract:Speech emotion recognition is a challenging task for three main reasons: 1) human emotion is abstract, which means it is hard to distinguish; 2) in general, human emotion can only be detected in some specific moments during a long utterance; 3) speech data with emotional labeling is usually limited. In this paper, we present a novel attention based fully convolutional network for speech emotion recognition. We employ fully convolutional network as it is able to handle variable-length speech, free of the demand of segmentation to keep critical information not lost. The proposed attention mechanism can make our model be aware of which time-frequency region of speech spectrogram is more emotion-relevant. Considering limited data, the transfer learning is also adapted to improve the accuracy. Especially, it's interesting to observe obvious improvement obtained with natural scene image based pre-trained model. Validated on the publicly available IEMOCAP corpus, the proposed model outperformed the state-of-the-art methods with a weighted accuracy of 70.4% and an unweighted accuracy of 63.9% respectively.

Sound,Machine Learning,Audio and Speech Processing
Mandarin Continuous Digit Speech Recognition System

xu haitian,wu ji,wang zuoying

DOI: https://doi.org/10.3321/j.issn:1002-8331.2002.02.034

2002-01-01

Abstract:Mandarin Continuous Digit Speech Recogniton takes up an important position in the development of speech recognition.In this paper,a system is implemented to reduce confusion.Then modeling improvement and audio speed controlling are introduced,which give our system good performance.

Emotion Recognition via Continuous Mandarin Speech

Exploring Spatio-Temporal Representations by Integrating Attention-based Bidirectional-LSTM-RNNs and FCNs for Speech Emotion Recognition

EmoEars: an emotion recognition system for mandarin speech

Cost-Sensitive Learning for Emotion Robust Speaker Recognition

Mandarin Emotion Recognition Combining Acoustic and Emotional Point Information

Emotion Recognition and Conversion for Mandarin Speech

Real-time Speech Emotion Recognition Based on Syllable-Level Feature Extraction

Emotion Recognition From Noisy Speech

Semi-Automatic Emotion Recognition From Chinese Text

Speech Emotion Recognition Based on Syllable-Level Feature Extraction

Speech Emotion Recognition Based on Formant Characteristics Feature Extraction and Phoneme Type Convergence.

Emotion Recognition From Speech and Text using Long Short-Term Memory

Research on Chinese Speech Emotion Recognition Based on Deep Neural Network and Acoustic Features

Manifolds Based Emotion Recognition in Speech.

MES-P: an Emotional Tonal Speech Dataset in Mandarin Chinese with Distal and Proximal Labels

English speech emotion recognition method based on speech recognition

Study on Emotion Feature Analysis and Recognition in Speech Signal: an Overview

Mandarin Emotion Recognition Based on Multifractal Theory Towards Human-Robot Interaction

A Combined Cepstral Distance Method for Emotional Speech Recognition

Attention Based Fully Convolutional Network for Speech Emotion Recognition

Mandarin Continuous Digit Speech Recognition System