Abstract:Zero-shot cross-lingual transfer utilizing multilingual LLMs has become a popular learning paradigm for low-resource languages with no labeled training data. However, for NLP tasks that involve fine-grained predictions on words and phrases, the performance of zero-shot cross-lingual transfer learning lags far behind supervised fine-tuning methods. Therefore, it is common to exploit translation and label projection to further improve the performance by (1) translating training data that is available in a high-resource language (e.g., English) together with the gold labels into low-resource languages, and/or (2) translating test data in low-resource languages to a high-source language to run inference on, then projecting the predicted span-level labels back onto the original test data. However, state-of-the-art marker-based label projection methods suffer from translation quality degradation due to the extra label markers injected in the input to the translation model. In this work, we explore a new direction that leverages constrained decoding for label projection to overcome the aforementioned issues. Our new method not only can preserve the quality of translated texts but also has the versatility of being applicable to both translating training and translating test data strategies. This versatility is crucial as our experiments reveal that translating test data can lead to a considerable boost in performance compared to translating only training data. We evaluate on two cross-lingual transfer tasks, namely Named Entity Recognition and Event Argument Extraction, spanning 20 languages. The results demonstrate that our approach outperforms the state-of-the-art marker-based method by a large margin and also shows better performance than other label projection methods that rely on external word alignment.

Cross-lingual Projections Between Languages from Different Families.

Semi-supervised Learning Framework for Cross-Lingual Projection

Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment

Constrained Decoding for Cross-lingual Label Projection

Enhancing Cross-lingual Sentence Embedding for Low-resource Languages with Word Alignment

Frustratingly Easy Label Projection for Cross-lingual Transfer

Improving In-context Learning of Multilingual Generative Language Models with Cross-lingual Alignment

Soft Cross-lingual Syntax Projection for Dependency Parsing.

Improving Bilingual Lexicon Induction on Distant Language Pairs

A Family of Fuzzy Orthogonal Projection Models for Monolingual and Cross-lingual Hypernymy Prediction

Cross-lingual Pseudo-Projected Expectation Regularization for Weakly Supervised Learning

Mongolian-Chinese Cross-Language Query Expansion Based on Cross-Language Word Vectors

Improving Word Embeddings via Combining with Complementary Languages.

Massively Parallel Cross-Lingual Learning in Low-Resource Target Language Translation

XTransplant: A Probe into the Upper Bound Performance of Multilingual Capability and Culture Adaptability in LLMs via Mutual Cross-lingual Feed-forward Transplantation

Noise removing based on n-gram representations and its applications to cross-lingual projection

Embedding Projection for Targeted Cross-Lingual Sentiment: Model Comparisons and a Real-World Study

A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning.

A Distributed Representation-Based Framework for Cross-Lingual Transfer Parsing.

Exploiting Common Characters in Chinese and Japanese to Learn Cross-Lingual Word Embeddings Via Matrix Factorization.

Bitext Name Tagging for Cross-lingual Entity Annotation Projection