Abstract:Slot filling is identifying contiguous spans of words in an utterance that correspond to certain parameters (i.e., slots) of a user request/query. Slot filling is one of the most important challenges in modern task-oriented dialog systems. Supervised learning approaches have proven effective at tackling this challenge, but they need a significant amount of labeled training data in a given domain. However, new domains (i.e., unseen in training) may emerge after deployment. Thus, it is imperative that these models seamlessly adapt and fill slots from both seen and unseen domains -- unseen domains contain unseen slot types with no training data, and even seen slots in unseen domains are typically presented in different contexts. This setting is commonly referred to as zero-shot slot filling. Little work has focused on this setting, with limited experimental evaluation. Existing models that mainly rely on context-independent embedding-based similarity measures fail to detect slot values in unseen domains or do so only partially. We propose a new zero-shot slot filling neural model, LEONA, which works in three steps. Step one acquires domain-oblivious, context-aware representations of the utterance word by exploiting (a) linguistic features; (b) named entity recognition cues; (c) contextual embeddings from pre-trained language models. Step two fine-tunes these rich representations and produces slot-independent tags for each word. Step three exploits generalizable context-aware utterance-slot similarity features at the word level, uses slot-independent tags, and contextualizes them to produce slot-specific predictions for each word. Our thorough evaluation on four diverse public datasets demonstrates that our approach consistently outperforms the SOTA models by 17.52%, 22.15%, 17.42%, and 17.95% on average for unseen domains on SNIPS, ATIS, MultiWOZ, and SGD datasets, respectively.

Improving Zero-shot Cross-domain Slot Filling Via Transformer-based Slot Semantics Fusion

Exploiting domain-slot related keywords description for Few-Shot Cross-Domain Dialogue State Tracking

Slot Transferability for Cross-domain Slot Filling.

Hierarchical Tagger with Multi-task Learning for Cross-domain Slot Filling

Cross-Domain Slot Filling As Machine Reading Comprehension

Exploring Transferability Measures and Domain Selection in Cross-Domain Slot Filling

Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling

Linguistically-Enriched and Context-Aware Zero-shot Slot Filling

A Two-Stage Selective Fusion Framework for Joint Intent Detection and Slot Filling

A CO-INTERACTIVE TRANSFORMER FOR JOINT SLOT FILLING AND INTENT DETECTION

HierarchicalContrast: A Coarse-to-Fine Contrastive Learning Framework for Cross-Domain Zero-Shot Slot Filling

Semantically Guided Enhanced Fusion for Intent Detection and Slot Filling

Zero-Shot Slot Filling Via Latent Question Representation And Reading Comprehension

A Novel Model Based on a Transformer for Intent Detection and Slot Filling

Deep Multi-task Learning with Cross Connected Layer for Slot Filling.

SEED: A Cross-Layer Semantic Enhanced SLU Model with Role Context Differentiated Fusion

Word embeddings: A semi-supervised learning method for slot-filling in spoken dialog systems

AISFG: Abundant Information Slot Filling Generator

Revisit Out-Of-Vocabulary Problem for Slot Filling: A Unified Contrastive Framework with Multi-Level Data Augmentations

SIAST: A Slot Imbalance-Aware Self-Training Scheme for Semi-Supervised Slot Filling

Intent-Slot Correlation Modeling for Joint Intent Prediction and Slot Filling