Abstract:Natural language understanding (NLU) is the task of semantic decoding of human languages by machines. NLU models rely heavily on large training data to ensure good performance. However, substantial languages and domains have very few data resources and domain experts. It is necessary to overcome the data scarcity challenge, when very few or even zero training samples are available. In this thesis, we focus on developing cross-lingual and cross-domain methods to tackle the low-resource issues. First, we propose to improve the model's cross-lingual ability by focusing on the task-related keywords, enhancing the model's robustness and regularizing the representations. We find that the representations for low-resource languages can be easily and greatly improved by focusing on just the keywords. Second, we present Order-Reduced Modeling methods for the cross-lingual adaptation, and find that modeling partial word orders instead of the whole sequence can improve the robustness of the model against word order differences between languages and task knowledge transfer to low-resource languages. Third, we propose to leverage different levels of domain-related corpora and additional masking of data in the pre-training for the cross-domain adaptation, and discover that more challenging pre-training can better address the domain discrepancy issue in the task knowledge transfer. Finally, we introduce a coarse-to-fine framework, Coach, and a cross-lingual and cross-domain parsing framework, X2Parser. Coach decomposes the representation learning process into a coarse-grained and a fine-grained feature learning, and X2Parser simplifies the hierarchical task structures into flattened ones. We observe that simplifying task structures makes the representation learning more effective for low-resource languages and domains.

TransAdv: A Translation-based Adversarial Learning Framework for Zero-Resource Cross-Lingual Named Entity Recognition.

Zero-Resource Multilingual Model Transfer: Learning What to Share

Cross-Lingual Named Entity Recognition Based on Attention and Adversarial Training

CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation

Cross lingual transfer learning for zero-resource domain adaptation

Neural Cross-Lingual Named Entity Recognition with Minimal Resources

UniTrans : Unifying Model Transfer and Data Transfer for Cross-Lingual Named Entity Recognition with Unlabeled Data

PRAM: An End-to-end Prototype-based Representation Alignment Model for Zero-resource Cross-lingual Named Entity Recognition

Research on Named Entity Recognition for Spoken Language Understanding Using Adversarial Transfer Learning

A Dual-Contrastive Framework for Low-Resource Cross-Lingual Named Entity Recognition

AdvPicker: Effectively Leveraging Unlabeled Data Via Adversarial Discriminator for Cross-Lingual NER

Cross-Lingual Transfer Robustness to Lower-Resource Languages on Adversarial Datasets

Adversarial Transfer Learning for Chinese Named Entity Recognition with Self-Attention Mechanism.

Domain-Invariant Feature Progressive Distillation with Adversarial Adaptive Augmentation for Low-Resource Cross-Domain NER.

Enhanced Meta-Learning for Cross-lingual Named Entity Recognition with Minimal Resources.

Zero-Shot Cross-Lingual Named Entity Recognition Via Progressive Multi-Teacher Distillation

A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning.

Effective Transfer Learning for Low-Resource Natural Language Understanding

Cross-Lingual Named Entity Recognition Using Parallel Corpus: A New Approach Using XLM-RoBERTa Alignment

Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-Resource Settings

Zero-Resource Cross-Domain Named Entity Recognition