Abstract:Most dominant neural machine translation (NMT) models are restricted to make predictions only according to the local context of preceding words in a left-to-right manner. Although many previous studies try to incorporate global information into NMT models, there still exist limitations on how to effectively exploit bidirectional global context. In this paper, we propose a Confidence Based Bidirectional Global Context Aware (CBBGCA) training framework for NMT, where the NMT model is jointly trained with an auxiliary conditional masked language model (CMLM). The training consists of two stages: (1) multi-task joint training; (2) confidence based knowledge distillation. At the first stage, by sharing encoder parameters, the NMT model is additionally supervised by the signal from the CMLM decoder that contains bidirectional global contexts. Moreover, at the second stage, using the CMLM as teacher, we further pertinently incorporate bidirectional global context to the NMT model on its unconfidently-predicted target words via knowledge distillation. Experimental results show that our proposed CBBGCA training framework significantly improves the NMT model by +1.02, +1.30 and +0.57 BLEU scores on three large-scale translation datasets, namely WMT'14 English-to-German, WMT'19 Chinese-to-English and WMT'14 English-to-French, respectively.

One Type Context is Not Enough: Global Context-Aware Neural Machine Translation

Towards Making the Most of Context in Neural Machine Translation

Context-Adaptive Document-Level Neural Machine Translation

Diving Deep into Context-Aware Neural Machine Translation

Learning Contextualized Sentence Representations for Document-Level Neural Machine Translation

Improving Context-Aware Neural Machine Translation with Source-side Monolingual Documents.

Context Modeling with Hierarchical Shallow Attention Structure for Document-Level NMT

SMDT: Selective Memory-Augmented Neural Document Translation

Exploiting Cross-Sentence Context for Neural Machine Translation

Confidence Based Bidirectional Global Context Aware Training Framework for Neural Machine Translation

Context-Aware Learning for Neural Machine Translation

Context Gates for Neural Machine Translation

HanoiT: Enhancing Context-aware Translation via Selective Context

Selective Memory-Augmented Document Translation with Diverse Global Context

A Case Study on Context-Aware Neural Machine Translation with Multi-Task Learning

Neural Machine Translation with Sentence-Level Topic Context

Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation

Document Graph for Neural Machine Translation

C L ] 1 0 Ju n 20 18 Deconvolution-Based Global Decoding for Neural Machine Translation

Document-level Neural Machine Translation with Document Embeddings

Breaking the Corpus Bottleneck for Context-Aware Neural Machine Translation with Cross-Task Pre-training.