Abstract:Speech interaction systems have been gaining popularity in recent years. The main purpose of these systems is to generate more satisfactory responses according to users' speech utterances, in which the most critical problem is to analyze user intention. Researches show that user intention conveyed through speech is not only expressed by content, but also closely related with users' speaking manners (e.g. with or without acoustic emphasis). How to incorporate these heterogeneous attributes to infer user intention remains an open problem. In this paper, we define Intention Prominence (IP) as the semantic combination of focus by text and emphasis by speech, and propose a multi-task deep learning framework to predict IP. Specifically, we first use long short-term memory (LSTM) which is capable of modeling long short-term contextual dependencies to detect focus and emphasis, and incorporate the tasks for focus and emphasis detection with multi-task learning (MTL) to reinforce the performance of each other. We then employ Bayesian network (BN) to incorporate multimodal features (focus, emphasis, and location reflecting users' dialect conventions) to predict IP based on feature correlations. Experiments on a data set of 135,566 utterances collected from real-world Sogou Voice Assistant illustrate that our method can outperform the comparison methods over 6.9-24.5% in terms of F1-measure. Moreover, a real practice in the Sogou Voice Assistant indicates that our method can improve the performance on user intention understanding by 7%.

Intention Recognition Based on Multi-layer Attention and Label Embedding

Learnable Hierarchical Label Embedding and Grouping for Visual Intention Understanding

HAIN: Multi-label Classification with Hierarchical Attention-based Interaction Network for Multi-turn Dialogue Texts

Few-Shot Intent Detection with Label-Enhanced Hierarchical Feature Learning and Graph Neural Networks

HAN-ReGRU: Hierarchical Attention Network with Residual Gated Recurrent Unit for Emotion Recognition in Conversation

Multi-Task Deep Learning for User Intention Understanding in Speech Interaction Systems

Label Hierarchical Structure-Aware Multi-Label Few-Shot Intent Detection Via Prompt Tuning

Balance the Labels: Hierarchical Label Structured Network for Dialogue Act Recognition

Label-Attentive Hierarchical Attention Network for Text Classification

A Deep Modular Label Attention Network for Multi-label Text Classification

Attention with Intention for a Neural Network Conversation Model

Jointly Trained Sequential Labeling and Classification by Sparse Attention Neural Networks.

LA-HCN: Label-based Attention for Hierarchical Multi-label TextClassification Neural Network

Dialogue Intent Classification with Long Short-Term Memory Networks.

Human Conversation Analysis Using Attentive Multimodal Networks with Hierarchical Encoder-Decoder

Multi-grained Label Refinement Network with Dependency Structures for Joint Intent Detection and Slot Filling

Semantic Enhancement Network Integrating Label Knowledge for Multi-modal Emotion Recognition

Multi-Point Semantic Representation for Intent Classification

HIPPL: Hierarchical Intent-Inferring Pointer Network With Pseudo Labeling for Consistent Persona-Driven Dialogue Generation

Hierarchical Network with Label Embedding for Contextual Emotion Recognition

Speech Emotion Recognition Via Multi-Level Attention Network