Multi-view fusion for universal translation quality estimation
Hui Huang,Shuangzhi Wu,Kehai Chen,Xinnian Liang,Hui Di,Muyun Yang,Tiejun Zhao
DOI: https://doi.org/10.1016/j.inffus.2023.102022
IF: 18.6
2023-09-20
Information Fusion
Abstract:Machine translation quality estimation (QE) aims to evaluate the result of translation without reference. Despite the progress it has made, state-of-the-art QE models are proven to be biased. More specifically, they over-rely on spurious statistical features while ignoring the bilingual semantic adequacy, leading to performance degradation. Besides, existing approaches require large amounts of annotation data, restricting their applications in new domains and languages. In this work, we propose a universal framework for quality estimation based on multi-view fusion. We first introduce noise to the target side of the parallel sentence pair, either by pre-trained language model or by large language model. After that, with the clean parallel pairs and the noised pairs as different views, the QE model is trained to distinguish the clean pairs from the noised ones. Our method can improve the accuracy and generalizability in supervised scenario, and can solely perform estimation in zero-shot scenario. We perform experiments on WMT QE evaluation datasets under different scenarios, verifying the effectiveness of our method. We also make an in-depth investigation of the bias of QE model.
computer science, artificial intelligence, theory & methods