Abstract:Slot tagging is a key component in a task-oriented dialogue system. Conversational agents need to understand human input by training on large amounts of annotated data. However, most human languages are low-resource and lack annotated training data for slot tagging task. Therefore, we aim to leverage cross-lingual transfer learning from high-resource languages to low-resource ones. In this paper, we propose an adversarial cross-lingual transfer model with multi-level language shared and specific knowledge to improve the slot tagging task of low-resource languages. Our method explicitly separates the model into the language-shared part and language-specific part to transfer language-independent knowledge. To refine shared knowledge in the latent space, we add a language discriminator and employ adversarial training to reinforce feature separation. Besides, we adopt a novel multi-level feature transfer in an incremental and progressive way to acquire multi-granularity shared knowledge. To mitigate the discrepancies between the feature distributions of language specific and shared knowledge, we propose the neural adapters to fuse features from different sources. Experiments show that our proposed model consistently outperforms monolingual baseline with a statistically significant margin up to 2.09%, even higher improvement of 12.21% in the zero-shot setting. Further analysis demonstrates that our method could effectively alleviate data scarcity of low-resource languages.

Adversarial Domain Adaptation for Cross-lingual Information Retrieval with Multilingual BERT

Cross-lingual Information Retrieval with BERT

Cross-Lingual Text Image Recognition Via Multi-Task Sequence to Sequence Learning.

Cross-Lingual Relevance Transfer for Document Retrieval

Cross-Lingual Training of Dense Retrievers for Document Retrieval

Narrowing the language gap: domain adaptation guided cross-lingual passage re-ranking

Cross-Lingual Training with Dense Retrieval for Document Retrieval

A Study of Cross-Lingual Ability and Language-specific Information in Multilingual BERT

Feature Aggregation in Zero-Shot Cross-Lingual Transfer Using Multilingual BERT.

Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsing

Transfer Learning Approaches for Building Cross-Language Dense Retrieval Models

Enhancing Model Performance in Multilingual Information Retrieval with Comprehensive Data Engineering Techniques

Mind the Gap: Cross-Lingual Information Retrieval with Hierarchical Knowledge Enhancement

Adversarial And Domain-Aware Bert For Cross-Domain Sentiment Analysis

Parameter-efficient Zero-shot Transfer for Cross-Language Dense Retrieval with Adapters

Distillation for Multilingual Information Retrieval

MULTI-LEVEL CONTRASTIVE LEARNING FOR CROSS-LINGUAL ALIGNMENT

Cross-lingual Alignment Methods for Multilingual BERT: A Comparative Study

Adversarial Cross-Lingual Transfer Learning for Slot Tagging of Low-Resource Languages

Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Model

C2BERT - Cross-contrast BERT for Chinese Biomedical Sentence Representation.