Abstract:Slot filling is identifying contiguous spans of words in an utterance that correspond to certain parameters (i.e., slots) of a user request/query. Slot filling is one of the most important challenges in modern task-oriented dialog systems. Supervised learning approaches have proven effective at tackling this challenge, but they need a significant amount of labeled training data in a given domain. However, new domains (i.e., unseen in training) may emerge after deployment. Thus, it is imperative that these models seamlessly adapt and fill slots from both seen and unseen domains -- unseen domains contain unseen slot types with no training data, and even seen slots in unseen domains are typically presented in different contexts. This setting is commonly referred to as zero-shot slot filling. Little work has focused on this setting, with limited experimental evaluation. Existing models that mainly rely on context-independent embedding-based similarity measures fail to detect slot values in unseen domains or do so only partially. We propose a new zero-shot slot filling neural model, LEONA, which works in three steps. Step one acquires domain-oblivious, context-aware representations of the utterance word by exploiting (a) linguistic features; (b) named entity recognition cues; (c) contextual embeddings from pre-trained language models. Step two fine-tunes these rich representations and produces slot-independent tags for each word. Step three exploits generalizable context-aware utterance-slot similarity features at the word level, uses slot-independent tags, and contextualizes them to produce slot-specific predictions for each word. Our thorough evaluation on four diverse public datasets demonstrates that our approach consistently outperforms the SOTA models by 17.52%, 22.15%, 17.42%, and 17.95% on average for unseen domains on SNIPS, ATIS, MultiWOZ, and SGD datasets, respectively.

Word embeddings: A semi-supervised learning method for slot-filling in spoken dialog systems

Using Word Confusion Networks for Slot Filling in Spoken Language Understanding.

Combining Word Feature Vector Method with the Convolutional Neural Network for Slot Filling in Spoken Language Understanding

Effective Slot Filling via Weakly-Supervised Dual-Model Learning

Recurrent Neural Networks with Pre-trained Language Model Embedding for Slot Filling Task

Prior Knowledge Driven Label Embedding for Slot Filling in Natural Language Understanding

Dial2vec: Self-Guided Contrastive Learning of Unsupervised Dialogue Embeddings

ESIE-BERT: Enriching Sub-words Information Explicitly with BERT for Joint Intent Classification and SlotFilling

Deep Multi-task Learning with Cross Connected Layer for Slot Filling.

Improving Automatic Speech Recognition and Speech Translation Via Word Embedding Prediction

Linguistically-Enriched and Context-Aware Zero-shot Slot Filling

Semi-Supervised Learning of Dialogue Acts Using Sentence Similarity Based on Word Embeddings

Pre-Trained Joint Model for Intent Classification and Slot Filling with Semantic Feature Fusion

Conversational Word Embedding for Retrieval-Based Dialog System

A Bi-model based RNN Semantic Frame Parsing Model for Intent Detection and Slot Filling

Slot Self-Attentive Dialogue State Tracking

Acoustic Word Embeddings for End-to-End Speech Synthesis

Learning Chinese Word Embeddings from Stroke, Structure and Pinyin of Characters

Improving the Generalized Performance of Deep Embedding for Text-Independent Speaker Verification

Improved Learning of Word Embeddings with Word Definitions and Semantic Injection.

A Self-Attentive Model with Gate Mechanism for Spoken Language Understanding