Abstract:Currently, in spoken language understanding (SLU) systems, the automatic speech recognition (ASR) module produces multiple interpretations (or hypotheses) for the input audio signal and the natural language understanding (NLU) module takes the one with the highest confidence score for domain or intent classification. However, the interpretations can be noisy, and solely relying on one interpretation can cause information loss. To address the problem, many research works attempt to rerank the interpretations for a better choice while some recent works get better performance by integrating all the hypotheses during prediction. In this paper, we follow the way of integrating hypotheses but strengthen the training mode by involving more tasks, some of which may be not in existing tasks of NLU but relevant, via multi-task learning or transfer learning. Moreover, we propose the Hierarchical Attention Mechanism (HAM) to further improve the performance with the acoustic-model features like confidence scores, which are ignored in the current hypotheses integration models. The experimental results show that compared to the standard estimation with one hypothesis, the multi-task learning with HAM can improve the domain and intent classification by relatively 19% and 37%, which are much higher than improvements with current integration or reranking methods. To illustrate the cause of improvements brought by our model, we decode the hidden representations of some utterance examples and compare the generated texts with hypotheses and transcripts. The comparison shows that our model could recover the transcription by integrating the fragmented information among hypotheses and identifying the frequent error patterns of the ASR module, and even rewrite the query for a better understanding, which reveals the characteristic of multi-task learning of broadcasting knowledge.

A Joint Multi-Task Learning Framework for Spoken Language Understanding

Towards Spoken Language Understanding via Multi-level Multi-grained Contrastive Learning

A Joint Learning Framework with BERT for Spoken Language Understanding.

A Self-Attentive Model with Gate Mechanism for Spoken Language Understanding

Joint Spoken Language Understanding And Domain Adaptive Language Modeling

A Joint and Domain-Adaptive Approach to Spoken Language Understanding

FC-MTLF: A Fine- and Coarse-grained Multi-Task Learning Framework for Cross-Lingual Spoken Language Understanding.

Conditional Joint Model For Spoken Dialogue System

A Unified Framework for Multi-intent Spoken Language Understanding with prompting

Deep Multi-task Learning with Cross Connected Layer for Slot Filling.

Multi-task Learning of Spoken Language Understanding by Integrating N-Best Hypotheses with Hierarchical Attention.

On joint training with interfaces for spoken language understanding

A Unified Approach to Nested and Non-Nested Slots for Spoken Language Understanding

A Self-Attention Joint Model for Spoken Language Understanding in Situational Dialog Applications

Position-aware Interactive Attention Network for Multi-Intent Spoken Language Understanding

Semi-Supervised Training Using Adversarial Multi-Task Learning for Spoken Language Understanding.

A Joint Model of Intent Determination and Slot Filling for Spoken Language Understanding.

Joint Modelling of Spoken Language Understanding Tasks with Integrated Dialog History

A Two-Stage Selective Fusion Framework for Joint Intent Detection and Slot Filling

Aligner²: Enhancing Joint Multiple Intent Detection and Slot Filling Via Adjustive and Forced Cross-Task Alignment

Spoken Language Understanding for Task-oriented Dialogue Systems with Augmented Memory Networks.