Abstract:Spoken Language Understanding (SLU), a core component of the task-oriented dialogue system, expects a shorter inference facing the impatience of human users. Existing work increases inference speed by designing non-autoregressive models for single-turn SLU tasks but fails to apply tomulti-turn SLU in confronting the dialogue history. The intuitive idea is to concatenate all historical utterances and utilize the non-autoregressive models directly. However, this approach seriously misses the salient historical information and suffers from the uncoordinated-slot problems. To overcome those shortcomings, we propose a novel model for multi-turn SLU named Salient History Attention with Layer-Refined Transformer (SHA-LRT), which comprises a SHA module, a Layer-Refined Mechanism (LRM), and a Slot Label Generation (SLG) task. SHA captures salient historical information for the current dialogue from both historical utterances and results via a well-designed history-attention mechanism. LRM predicts preliminary SLU results from Transformer's middle states and utilizes them to guide the final prediction, and SLG obtains the sequential dependency information for the non-autoregressive encoder. Experiments on public datasets indicate that our model significantly improves multi-turn SLU performance (17.5% on Overall) with accelerating (nearly 15 times) the inference process over the state-of-the-art baseline as well as effective on the single-turn SLU tasks.

Using Bidirectional Transformer-CRF for Spoken Language Understanding.

A CO-INTERACTIVE TRANSFORMER FOR JOINT SLOT FILLING AND INTENT DETECTION

On joint training with interfaces for spoken language understanding

A Joint and Domain-Adaptive Approach to Spoken Language Understanding

A Joint Multi-Task Learning Framework for Spoken Language Understanding

A Scope Sensitive and Result Attentive Model for Multi-Intent Spoken Language Understanding.

A Joint Model of Intent Determination and Slot Filling for Spoken Language Understanding.

A Typed Iteration Approach for Spoken Language Understanding

Bridging Speech and Textual Pre-trained Models with Unsupervised ASR.

A Model with Length-Variable Attention for Spoken Language Understanding

CIF-PT: Bridging Speech and Text Representations for Spoken Language Understanding via Continuous Integrate-and-Fire Pre-Training

Incorporating Complete Syntactical Knowledge for Spoken Language Understanding

SASGBC

Jointly Encoding Word Confusion Network and Dialogue Context with BERT for Spoken Language Understanding

End-to-End Cross-Lingual Spoken Language Understanding Model with Multilingual Pretraining.

Multi-Classification Model for Spoken Language Understanding

A Self-Attentive Model with Gate Mechanism for Spoken Language Understanding

Capture Salient Historical Information: A Fast and Accurate Non-autoregressive Model for Multi-turn Spoken Language Understanding

Using Word Confusion Networks for Slot Filling in Spoken Language Understanding.

Deep belief network based CRF for spoken language understanding

A Unified Framework for Multi-intent Spoken Language Understanding with prompting