Learning Policies for Multilingual Training of Neural Machine Translation Systems

Gaurav Kumar,Philipp Koehn,Sanjeev Khudanpur
DOI: https://doi.org/10.48550/arXiv.2103.06964
2021-03-12
Abstract:Low-resource Multilingual Neural Machine Translation (MNMT) is typically tasked with improving the translation performance on one or more language pairs with the aid of high-resource language pairs. In this paper, we propose two simple search based curricula -- orderings of the multilingual training data -- which help improve translation performance in conjunction with existing techniques such as fine-tuning. Additionally, we attempt to learn a curriculum for MNMT from scratch jointly with the training of the translation system with the aid of contextual multi-arm bandits. We show on the FLORES low-resource translation dataset that these learned curricula can provide better starting points for fine tuning and improve overall performance of the translation system.
Computation and Language
What problem does this paper attempt to address?