Abstract:Zero-shot dialogue state tracking (DST) seeks to enable dialogue systems to transition to unfamiliar domains without manual annotation or extensive retraining. Prior research has approached this objective by embedding prompts into language models (LMs). Common methodologies include integrating prompts at the input layer or introducing learnable variables at each transformer layer. Nonetheless, each strategy exhibits inherent limitations. Prompts integrated at the input layer risk underutilization, with their impact potentially diminishing across successive transformer layers. Conversely, the addition of learnable variables to each layer can complicate the training process and increase inference latency. To tackle the issues mentioned above, this paper proposes Dual Low-Rank Adaptation (DualLoRA), a plug-and-play architecture designed for zero-shot DST. DualLoRA incorporates two distinct Low-Rank Adaptation (LoRA) components, targeting both dialogue context processing and prompt optimization, to ensure the comprehensive influence of prompts throughout the transformer model layers. This is achieved without incurring additional inference latency, showcasing an efficient integration into existing architectures. Through rigorous evaluation on the MultiWOZ and SGD datasets, DualLoRA demonstrates notable improvements across multiple domains, outperforming traditional baseline methods in zero-shot settings. Our code is accessible at: \url{<a class="link-external link-https" href="https://github.com/suntea233/DualLoRA" rel="external noopener nofollow">this https URL</a>}.

Improving Zero-Shot Cross-Lingual Dialogue State Tracking Via Contrastive Learning

Zero-shot language extension for dialogue state tracking via pre-trained models and multi-auxiliary-tasks fine-tuning

Exploiting domain-slot related keywords description for Few-Shot Cross-Domain Dialogue State Tracking

Zero-Shot Cross-Domain Dialogue State Tracking via Dual Low-Rank Adaptation

ECO-DST: An Efficient Cross-lingual Dialogue State Tracking Framework

Semantic Parsing by Large Language Models for Intricate Updating Strategies of Zero-Shot Dialogue State Tracking

Attention-Informed Mixed-Language Training for Zero-Shot Cross-Lingual Task-Oriented Dialogue Systems

UNO-DST: Leveraging Unlabelled Data in Zero-Shot Dialogue State Tracking

Cross-lingual Intermediate Fine-tuning improves Dialogue State Tracking

XQA-DST: Multi-Domain and Multi-Lingual Dialogue State Tracking

A Zero-Shot Open-Vocabulary Pipeline for Dialogue Understanding

ChatZero:Zero-shot Cross-Lingual Dialogue Generation via Pseudo-Target Language

Large Language Models as Zero-shot Dialogue State Tracker through Function Calling

Diverse and Effective Synthetic Data Generation for Adaptable Zero-Shot Dialogue State Tracking

Non-Autoregressive Dialog State Tracking

Intent-driven In-context Learning for Few-shot Dialogue State Tracking

Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking

Enhanced Multi-Domain Dialogue State Tracker with Second-Order Slot Interactions

Enhancing Dialogue State Tracking Models through LLM-backed User-Agents Simulation

Benchmarks Underestimate the Readiness of Multi-lingual Dialogue Agents

Stabilized In-Context Learning with Pre-trained Language Models for Few Shot Dialogue State Tracking