Abstract:Despite substantial progress in multilingual extractive Question Answering (QA), models with high and uniformly distributed performance across languages remain challenging, especially for languages with limited resources. We study cross-lingual transfer mainly focusing on the Generalized Cross-Lingual Transfer (G-XLT) task, where the question language differs from the context language - a challenge that has received limited attention thus far. Our approach seeks to enhance cross-lingual QA transfer using a high-performing multilingual model trained on a large-scale dataset, complemented by a few thousand aligned QA examples across languages. Our proposed strategy combines cross-lingual sampling and advanced self-distillation training in generations to tackle the previous challenge. Notably, we introduce the novel mAP@k coefficients to fine-tune self-knowledge distillation loss, dynamically regulating the teacher's model knowledge to perform a balanced and effective knowledge transfer. We extensively evaluate our approach to assess XLT and G-XLT capabilities in extractive QA. Results reveal that our self-knowledge distillation approach outperforms standard cross-entropy fine-tuning by a significant margin. Importantly, when compared to a strong baseline that leverages a sizeable volume of machine-translated data, our approach shows competitive results despite the considerable challenge of operating within resource-constrained settings, even in zero-shot scenarios. Beyond performance improvements, we offer valuable insights through comprehensive analyses and an ablation study, further substantiating the benefits and constraints of our approach. In essence, we propose a practical solution to improve cross-lingual QA transfer by leveraging a few data resources in an efficient way.

Towards Zero-Shot Multilingual Synthetic Question and Answer Generation for Cross-Lingual Reading Comprehension

PAXQA: Generating Cross-lingual Question Answering Examples at Training Scale

Cross-lingual Transfer for Automatic Question Generation by Learning Interrogative Structures in Target Languages

Cross-Lingual Transfer Learning for Question Answering

Improving Zero-Shot Cross-lingual Transfer for Multilingual Question Answering over Knowledge Graph

Promoting Generalized Cross-lingual Question Answering in Few-resource Scenarios via Self-knowledge Distillation

Training Question Answering Models From Synthetic Data

xGQA: Cross-Lingual Visual Question Answering

Bridging the Language Gap: Knowledge Injected Multilingual Question Answering

GeMQuAD : Generating Multilingual Question Answering Datasets from Large Language Models using Few Shot Learning

Few-Shot Data Synthesis for Open Domain Multi-Hop Question Answering

ZusammenQA: Data Augmentation with Specialized Models for Cross-lingual Open-retrieval Question Answering System

Learning to Answer Multilingual and Code-Mixed Questions

MaXM: Towards Multilingual Visual Question Answering

Cross-Lingual Question Answering over Knowledge Base as Reading Comprehension

Generalizing Visual Question Answering from Synthetic to Human-Written Questions via a Chain of QA with a Large Language Model

Understanding and Improving Zero-shot Multi-hop Reasoning in Generative Question Answering

XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answering

Cross-Lingual Training for Automatic Question Generation

QADYNAMICS: Training Dynamics-Driven Synthetic QA Diagnostic for Zero-Shot Commonsense Question Answering

Synthetic Multimodal Question Generation