LaDA: Latent Dialogue Action For Zero-shot Cross-lingual Neural Network Language Modeling

Zhanyu Ma,Jian Ye,Shuang Cheng
2023-08-05
Abstract:Cross-lingual adaptation has proven effective in spoken language understanding (SLU) systems with limited resources. Existing methods are frequently unsatisfactory for intent detection and slot filling, particularly for distant languages that differ significantly from the source language in scripts, morphology, and syntax. Latent Dialogue Action (LaDA) layer is proposed to optimize decoding strategy in order to address the aforementioned issues. The model consists of an additional layer of latent dialogue action. It enables our model to improve a system's capability of handling conversations with complex multilingual intent and slot values of distant languages. To the best of our knowledge, this is the first exhaustive investigation of the use of latent variables for optimizing cross-lingual SLU policy during the decode stage. LaDA obtains state-of-the-art results on public datasets for both zero-shot and few-shot adaptation.
Computation and Language
What problem does this paper attempt to address?
The paper mainly addresses the challenges of cross-lingual adaptation in low-resource spoken language understanding systems, particularly the poor performance of intent detection and slot filling tasks in target languages that have significant grammatical, morphological, or syntactic differences from the source language. The paper proposes a new model architecture called the Latent Dialogue Act Layer (LaDA), which aims to resolve the conflicts between intent and slot information in cross-lingual environments and improve the performance of spoken language understanding tasks (including intent detection and slot filling) in few-shot and zero-shot scenarios. LaDA optimizes the decoding strategy by introducing an additional latent dialogue act layer, enabling the model to better handle complex multilingual intents and slot values, especially in cases where there are significant grammatical differences between languages. Specifically, the LaDA model combines traditional language modeling with the latent dialogue act layer, which can generate independent Bernoulli distributions to represent the latent dialogue act of each word. Experimental results show that LaDA significantly improves the accuracy of intent detection and slot filling in various languages (such as German, Spanish, French, Hindi, and Thai), achieving state-of-the-art results in zero-shot learning settings. Additionally, the paper conducts ablation studies to verify the importance of each component and demonstrates the efficiency of LaDA in inference speed.