A Simple Baseline to Semi-Supervised Domain Adaptation for Machine Translation

Di Jin,Zhijing Jin,Joey Tianyi Zhou,Peter Szolovits

DOI: https://doi.org/10.48550/arXiv.2001.08140

2020-06-06

Abstract:State-of-the-art neural machine translation (NMT) systems are data-hungry and perform poorly on new domains with no supervised data. As data collection is expensive and infeasible in many cases, domain adaptation methods are needed. In this work, we propose a simple but effect approach to the semi-supervised domain adaptation scenario of NMT, where the aim is to improve the performance of a translation model on the target domain consisting of only non-parallel data with the help of supervised source domain data. This approach iteratively trains a Transformer-based NMT model via three training objectives: language modeling, back-translation, and supervised translation. We evaluate this method on two adaptation settings: adaptation between specific domains and adaptation from a general domain to specific domains, and on two language pairs: German to English and Romanian to English. With substantial performance improvement achieved---up to +19.31 BLEU over the strongest baseline, and +47.69 BLEU improvement over the unadapted model---we present this method as a simple but tough-to-beat baseline in the field of semi-supervised domain adaptation for NMT.

Computation and Language,Machine Learning