Abstract:Despite substantial progress in multilingual extractive Question Answering (QA), models with high and uniformly distributed performance across languages remain challenging, especially for languages with limited resources. We study cross-lingual transfer mainly focusing on the Generalized Cross-Lingual Transfer (G-XLT) task, where the question language differs from the context language - a challenge that has received limited attention thus far. Our approach seeks to enhance cross-lingual QA transfer using a high-performing multilingual model trained on a large-scale dataset, complemented by a few thousand aligned QA examples across languages. Our proposed strategy combines cross-lingual sampling and advanced self-distillation training in generations to tackle the previous challenge. Notably, we introduce the novel mAP@k coefficients to fine-tune self-knowledge distillation loss, dynamically regulating the teacher's model knowledge to perform a balanced and effective knowledge transfer. We extensively evaluate our approach to assess XLT and G-XLT capabilities in extractive QA. Results reveal that our self-knowledge distillation approach outperforms standard cross-entropy fine-tuning by a significant margin. Importantly, when compared to a strong baseline that leverages a sizeable volume of machine-translated data, our approach shows competitive results despite the considerable challenge of operating within resource-constrained settings, even in zero-shot scenarios. Beyond performance improvements, we offer valuable insights through comprehensive analyses and an ablation study, further substantiating the benefits and constraints of our approach. In essence, we propose a practical solution to improve cross-lingual QA transfer by leveraging a few data resources in an efficient way.

CLICKER: Attention-Based Cross-Lingual Commonsense Knowledge Transfer

Leveraging Knowledge in Multilingual Commonsense Reasoning

Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning Across Languages

Cross-Lingual Multi-Hop Knowledge Editing -- Benchmarks, Analysis and a Simple Contrastive Learning based Approach

Cross-Lingual Transfer for Natural Language Inference via Multilingual Prompt Translator

Sharing, Teaching and Aligning: Knowledgeable Transfer Learning for Cross-Lingual Machine Reading Comprehension

Cross-Lingual Question Answering over Knowledge Base as Reading Comprehension

Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoning

Breaking the Language Barrier: Improving Cross-Lingual Reasoning with Structured Self-Attention

Bridging the Language Gap: Knowledge Injected Multilingual Question Answering

Improving Commonsense Question Answering by Graph-based Iterative Retrieval over Multiple Knowledge Sources

Improving Low-resource Reading Comprehension via Cross-lingual Transposition Rethinking

LINKED: Eliciting, Filtering and Integrating Knowledge in Large Language Model for Commonsense Reasoning

XL-NBT: A Cross-lingual Neural Belief Tracking Framework

Promoting Generalized Cross-lingual Question Answering in Few-resource Scenarios via Self-knowledge Distillation

KEPR: Knowledge Enhancement and Plausibility Ranking for Generative Commonsense Question Answering

Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting

Cross-lingual Lifelong Learning

Improving Cross-lingual Representation for Semantic Retrieval with Code-switching

Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation

xCoT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoning