Abstract:Multi-source cross-lingual transfer aims to acquire task knowledge from multiple labelled source languages and transfer it to an unlabelled target language, enabling effective performance in this target language. The existing methods mainly focus on weighting predictions of language-specific classifiers trained in source languages to derive final results for target samples. However, we argue that, due to the language gap, language-specific classifiers inevitably generate many noisy predictions for target samples. Furthermore, these methods disregard the mutual guidance and utilization of knowledge among multiple source languages. To address these issues, we propose a novel model, Den-ML, which improves the model’s performance in multi-source scenarios through two perspectives: reducing prediction noise of language-specific classifiers and prompting mutual learning among these classifiers. Firstly, Den-ML devises a discrepancy-guided denoising learning method to learn discriminative representations for the target language, thus mitigating the noise prediction of classifiers. Secondly, Den-ML develops a pseudo-label-supervised mutual learning method, which relies on forcing probability distribution interactions among multiple language-specific classifiers for knowledge transfer, thus achieving mutual learning among classifiers. We conduct experiments on three different tasks, named entity recognition, paraphrase identification and natural language inference, with two different multi-source combination settings (same- and different-family settings) covering 39 languages. Our approach outperforms the benchmark and the SOTA model in most metrics for all three tasks in different settings. In addition, we perform ablation, visualization and analysis experiments on three different tasks, and the experimental results validate the effectiveness of our approach.

Learn from Your Neighbor: Learning Multi-modal Mappings from Sparse Annotations

Transfer and Multi-Task Learning for Noun-Noun Compound Interpretation

Multi-Label Transfer Learning with Sparse Representation

Learning from Semantically Dependent Multi-Tasks

New Graph Structured Sparsity Model for Multi-label Image Annotations

Multi-Task Learning for Joint Semantic Role and Proto-Role Labeling

Graph Regularized Low-Rank Feature Mapping for Multi-Label Learning with Application to Image Annotation

Learning Neighborhood Representation from Multi-Modal Multi-Graph: Image, Text, Mobility Graph and Beyond

On the Limits of Multi-modal Meta-Learning with Auxiliary Task Modulation Using Conditional Batch Normalization

Curriculum-Based Neighborhood Sampling For Sequence Prediction

Semi-supervised Multimodal Representation Learning through a Global Workspace

Multi-annotator Deep Learning: A Probabilistic Framework for Classification

Preserving Modality Structure Improves Multi-Modal Learning

Towards Multi-Semantic Image Annotation with Graph Regularized Exclusive Group Lasso

MUCH: Mutual Coupling Enhancement of Scene Recognition and Dense Captioning

Deep Multi-Modal Sets

Neural Task Representations as Weak Supervision for Model Agnostic Cross-Lingual Transfer

Den-ML: Multi-source Cross-Lingual Transfer Via Denoising Mutual Learning

Finding and Editing Multi-Modal Neurons in Pre-Trained Transformers

Exploring transfer learning for Deep NLP systems on rarely annotated languages

Learning Multiple Dense Prediction Tasks from Partially Annotated Data