Findings of the 2016 WMT Shared Task on Cross-lingual Pronoun Prediction

Liane Guillou,Christian Hardmeier,Preslav Nakov,Sara Stymne,Jörg Tiedemann,Yannick Versley,Mauro Cettolo,Bonnie Webber,Andrei Popescu-Belis

DOI: https://doi.org/10.48550/arXiv.1911.12091

2019-11-27

Computation and Language

Abstract:We describe the design, the evaluation setup, and the results of the 2016 WMT shared task on cross-lingual pronoun prediction. This is a classification task in which participants are asked to provide predictions on what pronoun class label should replace a placeholder value in the target-language text, provided in lemmatised and PoS-tagged form. We provided four subtasks, for the English-French and English-German language pairs, in both directions. Eleven teams participated in the shared task; nine for the English-French subtask, five for French-English, nine for English-German, and six for German-English. Most of the submissions outperformed two strong language-model based baseline systems, with systems using deep recurrent neural networks outperforming those using other architectures for most language pairs.