Inferring Emotions from Large-Scale Internet Voice Data.
Jia,Suping Zhou,Yufeng Yin,Boya Wu,Wei Chen,Fanbo Meng,Yanfeng Wang
DOI: https://doi.org/10.1109/tmm.2018.2887016
IF: 7.3
2018-01-01
IEEE Transactions on Multimedia
Abstract:As voice dialog applications (VDAs, e.g., Siri,11http://www.apple.com/ios/siri/.Cortana,22http://www.microsoft.com/en-us/mobile/campaign-cortana/.Google Now33http://www.google.com/landing/now/.) are increasing in popularity, inferring emotions from the large-scale internet voice data generated from VDAs can help give a more reasonable and humane response. However, the tremendous amounts of users in large-scale internet voice data lead to a great diversity of users accents and expression patterns. Therefore, the traditional speech emotion recognition methods, which mainly target acted corpora, cannot effectively handle the massive and diverse amount of internet voice data. To address this issue, we carry out a series of observations, find suitable emotion categories for large-scale internet voice data, and verify the indicators of the social attributes (query time, query topic, and users location) and emotion inferring. Based on our observations, two different strategies are employed to solve the problem. First, a deep sparse neural network model that uses acoustic information, textual information, and three indicators (a temporal indicator, descriptive indicator, and geo-social indicator) as the input is proposed. Then, to capture the contextual information, we propose a hybrid emotion inference model that includes long short-term memory to capture the acoustic features and a latent dirichlet allocation to extract text features. Experiments on 93 000 utterances collected from the Sogou Voice Assistant44http://yy.sogou.com.(Chinese Siri) validate the effectiveness of the proposed methodologies. Furthermore, we compare the two methodologies and give their advantages and disadvantages.