Abstract:Spoken Language Understanding (SLU) aims to extract structured information from speech recognized texts, which suffers from inaccurate automatic speech recognition (ASR) (especially in a specific dialogue domain). Language Modeling (LM) is significant to ASR for producing natural sentences. To improve the SLU performance in a specific domain, we try two ways: (1) domain adaptive language modeling which tends to recognize in-domain utterances; (2) joint modeling of SLU and LM which distills semantic information to build semantic-aware LM, and also helps SLU by semi-supervised learning (LM is unsupervised). To unify these two approaches, we propose a multi-task model (MTM) that jointly performs two SLU tasks (slot filling and intent detection), domain-specific LM and domain-free (general) LM. In the proposed multi-task architecture, the shared-private network is utilized to automatically learn which part of general data can be shared by the specific domain or not. We attempt to further improve the SLU and ASR performance in a specific domain with a few of labeled data in the specific domain and plenty of unlabeled data in general domain. The experiments show that the proposed MTM can obtain 4.06% absolute WER (Word Error Rate) reduction in a car navigation domain, compared to a general-domain LM. For language understanding, the MTM outperforms the baseline (especially slot filling task) on the manual transcript, ASR 1-best output. By exploiting the domain-adaptive LM to rescore ASR output, our proposed model achieves further improvement in SLU (7.08% absolute F1 increase of slot filling task).

Conditional Joint Model For Spoken Dialogue System

A Self-Attention Joint Model for Spoken Language Understanding in Situational Dialog Applications

Joint Modelling of Spoken Language Understanding Tasks with Integrated Dialog History

End-to-End Joint Learning of Natural Language Understanding and Dialogue Manager

A Self-Attentive Model with Gate Mechanism for Spoken Language Understanding

A Joint and Domain-Adaptive Approach to Spoken Language Understanding

A Model with Length-Variable Attention for Spoken Language Understanding

Joint Spoken Language Understanding And Domain Adaptive Language Modeling

Towards Joint Modeling of Dialogue Response and Speech Synthesis based on Large Language Model

Towards Spoken Language Understanding via Multi-level Multi-grained Contrastive Learning

Learning Dialogue History for Spoken Language Understanding.

Joint Dual Learning with Mutual Information Maximization for Natural Language Understanding and Generation in Dialogues

Multijugate Dual Learning for Low-Resource Task-Oriented Dialogue System.

Toward Joint Language Modeling for Speech Units and Text

Pre-Trained Joint Model for Intent Classification and Slot Filling with Semantic Feature Fusion

WORD-CLASS STOCHASTIC MODEL IN A SPOKEN LANGUAGE DIALOGUE SYSTEM

Joint Intent Detection Model for Task-oriented Human-Computer Dialogue System Using Asynchronous Training

A Joint Model of Intent Determination and Slot Filling for Spoken Language Understanding.

Large Language Model based Situational Dialogues for Second Language Learning

Retrieval Augmented End-to-End Spoken Dialog Models

ML-LMCL: Mutual Learning and Large-Margin Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding