Abstract:Effective emotion inference from user queries helps to give a more personified response for Voice Dialogue Applications(VDAs). The tremendous amounts of VDA users bring in diverse emotion expressions. How to achieve a high emotion inferring performance from large-scale Internet Voice Data in VDAs? Traditionally, researches on speech emotion recognition are based on acted voice datasets, which have limited speakers but strong and clear emotion expressions. Inspired by this, in this paper, we propose a novel approach to leverage acted voice data with strong emotion expressions to enhance large-scale unlabeled internet voice data with diverse emotion expressions for emotion inferring. Specifically, we propose a novel semi-supervised multi-modal curriculum augmentation deep learning framework. First, to learn more general emotion cues, we adopt a curriculum learning based epoch-wise training strategy, which trains our model guided by strong and balanced emotion samples from acted voice data and sub-sequently leverages weak and unbalanced emotion samples from internet voice data.Second, to employ more diverse emotion expressions, we design a Multi-path Mix-match Multimodal Deep Neural Network(MMMD), which effectively learns feature representations for multiple modalities and trains labeled and unlabeled data in hybrid semi-supervised methods for superior generalization and robustness. Experiments on an internet voice dataset with 500,000 utterances show our method outperforms (+10.09% in terms of F1) several alternative baselines, while an acted corpus with 2,397 utterances contributes 4.35%. To further compare our method with state-of-the-art techniques in traditionally acted voice datasets, we also conduct experiments on public dataset IEMOCAP. The results reveal the effectiveness of the proposed approach.

Acoustics, Content and Geo-Information Based Sentiment Prediction from Large-Scale Networked Voice Data

Audio Sentiment Analysis by Heterogeneous Signal Features Learned from Utterance-Based Parallel Neural Network.

Utterance-Based Audio Sentiment Analysis Learned by a Parallel Combination of CNN and LSTM.

Learning to Infer Public Emotions from Large-Scale Networked Voice Data

Inferring Emotions from Large-Scale Internet Voice Data.

Exploring Spatio-Temporal Representations by Integrating Attention-based Bidirectional-LSTM-RNNs and FCNs for Speech Emotion Recognition

Self-attention Transfer Networks for Speech Emotion Recognition

Hybrid Network Feature Extraction for Depression Assessment from Speech

Emotion Inferring from Large-scale Internet Voice Data: A Multimodal Deep Learning Approach

Learning Robust Heterogeneous Signal Features from Parallel Neural Network for Audio Sentiment Analysis

Inferring Users' Emotions For Human-Mobile Voice Dialogue Applications

Inferring Emotion from Conversational Voice Data: A Semi-Supervised Multi-Path Generative Neural Network Approach.

Inferring Emotion from Large-scale Internet Voice Data: A Semi-supervised Curriculum Augmentation Based Deep Learning Approach

Research on Chinese Speech Emotion Recognition Based on Deep Neural Network and Acoustic Features

Cross-Lingual Propagation for Deep Sentiment Analysis

A Deep Learning System for Sentiment Analysis of Service Calls

Artificial Intelligence Technology-Based Semantic Sentiment Analysis on Network Public Opinion Texts

Combining Vector Space Features and Convolution Neural Network for Text Sentiment Analysis.

Network Public Opinion Risk Prediction and Judgment Based on Deep Learning: A Model of Text Sentiment Analysis

Personalized language modeling by crowd sourcing with social network data for voice access of cloud applications

Deep Learning-Based Natural Language Processing Methods for Sentiment Analysis in Social Networks