Abstract:The majority of previous researches addressing multi-lingual IE are limited to zero-shot cross-lingual single-transfer (one-to-one) setting, with high-resource languages predominantly as source training data. As a result, these works provide little understanding and benefit for the realistic goal of developing a multi-lingual IE system that can generalize to as many languages as possible. Our study aims to fill this gap by providing a detailed analysis on Cross-Lingual Multi-Transferability (many-to-many transfer learning), for the recent IE corpora that cover a diverse set of languages. Specifically, we first determine the correlation between single-transfer performance and a wide range of linguistic-based distances. From the obtained insights, a combined language distance metric can be developed that is not only highly correlated but also robust across different tasks and model scales. Next, we investigate the more general zero-shot multi-lingual transfer settings where multiple languages are involved in the training and evaluation processes. Language clustering based on the newly defined distance can provide directions for achieving the optimal cost-performance trade-off in data (languages) selection problem. Finally, a relational-transfer setting is proposed to further incorporate multi-lingual unlabeled data based on adversarial training using the relation induced from the above linguistic distance.

Realistic Zero-Shot Cross-Lingual Transfer in Legal Topic Classification

Zero-Shot Cross-Lingual Transfer in Legal Domain Using Transformer Models

Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model

Cross-lingual Transfer of Monolingual Models

An Efficient Approach for Studying Cross-Lingual Transfer in Multilingual Language Models

An Empirical Study on Cross-X Transfer for Legal Judgment Prediction

Cross-Lingual Transfer in Zero-Shot Cross-Language Entity Linking

Towards Zero-shot Cross-lingual Image Retrieval and Tagging

A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning.

Zero-shot Cross-lingual Transfer Learning with Multiple Source and Target Languages for Information Extraction: Language Selection and Adversarial Training

Cross-lingual Pre-training Based Transfer for Zero-shot Neural Machine Translation.

Translation Artifacts in Cross-lingual Transfer Learning

Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-Resource Settings

Adaptive Cross-lingual Text Classification through In-Context One-Shot Demonstrations

Zero-Shot Cross-Lingual Sentiment Classification under Distribution Shift: an Exploratory Study

Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language Models

Language-Driven Cross-Modal Classifier for Zero-Shot Multi-Label Image Recognition

Modular Speech-to-Text Translation for Zero-Shot Cross-Modal Transfer

Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsing

Multi-granular Legal Topic Classification on Greek Legislation

Evaluating and explaining training strategies for zero-shot cross-lingual news sentiment analysis