Abstract:Translation Quality Estimation is critical to reducing post-editing efforts in machine translation and to cross-lingual corpus cleaning. As a research problem, quality estimation (QE) aims to directly estimate the quality of translation in a given pair of source and target sentences, and highlight the words that need corrections, without referencing to golden translations. In this paper, we propose Verdi, a novel framework for word-level and sentence-level post-editing effort estimation for bilingual corpora. Verdi adopts two word predictors to enable diverse features to be extracted from a pair of sentences for subsequent quality estimation, including a transformer-based neural machine translation (NMT) model and a pre-trained cross-lingual language model (XLM). We exploit the symmetric nature of bilingual corpora and apply model-level dual learning in the NMT predictor, which handles a primal task and a dual task simultaneously with weight sharing, leading to stronger context prediction ability than single-direction NMT models. By taking advantage of the dual learning scheme, we further design a novel feature to directly encode the translated target information without relying on the source context. Extensive experiments conducted on WMT20 QE tasks demonstrate that our method beats the winner of the competition and outperforms other baseline methods by a great margin. We further use the sentence-level scores provided by Verdi to clean a parallel corpus and observe benefits on both model performance and training efficiency.

NJUNLP's Submission for CCMT20 Quality Estimation Task.

NJUNLP’s Submission for CCMT 2023 Quality Estimation Task

NJU's submission to the WMT20 QE Shared Task.

NJU Submissions for the WMT19 Quality Estimation Shared Task

Submissions for the WMT 19 Quality Estimation Shared Task

Unify word-level and span-level tasks: NJUNLP's Participation for the WMT2023 Quality Estimation Shared Task

Alibaba-Translate China's Submission for WMT 2022 Quality Estimation Shared Task

NJUNLP’s Machine Translation System for CCMT-2020 Uighur $$\rightarrow $$ Chinese Translation Task

NJUNLP’s Machine Translation System for CCMT-2020 Uighur → Chinese Translation Task

SurreyAI 2023 Submission for the Quality Estimation Shared Task

Verdi: Quality Estimation and Error Detection for Bilingual Corpora

Beyond Glass-Box Features: Uncertainty Quantification Enhanced Quality Estimation for Neural Machine Translation

Self-Supervised Quality Estimation for Machine Translation.

Ensemble-based Transfer Learning for Low-resource Machine Translation Quality Estimation

Multi-view fusion for universal translation quality estimation

QE-EBM: Using Quality Estimators as Energy Loss for Machine Translation

Denoising Pre-training for Machine Translation Quality Estimation with Curriculum Learning.

Don't Rank, Combine! Combining Machine Translation Hypotheses Using Quality Estimation

Unsupervised Quality Estimation for Neural Machine Translation

HW-TSC's Participation at WMT 2021 Quality Estimation Shared Task.

DeepSubQE: Quality estimation for subtitle translations