Abstract:Recently, researchers have shown an increasing interest in incorporating linguistic knowledge into neural machine translation (NMT). To this end, previous works choose either to alter the architecture of NMT encoder to incorporate syntactic information into the translation model, or to generalize the embedding layer of the encoder to encode additional linguistic features. The former approach mainly focuses on injecting the syntactic structure of the source sentence into the encoding process, leading to a complicated model that lacks the flexibility to incorporate other types of knowledge. The latter extends word embeddings by considering additional linguistic knowledge as features to enrich the word representation. It thus does not explicitly balance the contribution from word embeddings and the contribution from additional linguistic knowledge. To address these limitations, this paper proposes a knowledge-aware NMT approach that models additional linguistic features in parallel to the word feature. The core idea is that we propose modeling a series of linguistic features at the word level (knowledge block) using a recurrent neural network (RNN). And in sentence level, those word-corresponding feature blocks are further encoded using a RNN encoder. In decoding, we propose a knowledge gate and an attention gate to dynamically control the proportions of information contributing to the generation of target words from different sources. Extensive experiments show that our approach is capable of better accounting for importance of additional linguistic, and we observe significant improvements from 1.0 to 2.3 BLEU points on Chinese$\leftrightarrow$ English and English$\rightarrow$German translation tasks.

Language-aware Interlingua for Multilingual Neural Machine Translation.

Mutual Information and Diverse Decoding Improve Neural Machine Translation.

Towards Interlingua Neural Machine Translation

High-resource Language-specific Training for Multilingual Neural Machine Translation.

Improving Zero-shot Neural Machine Translation on Language-specific Encoders-Decoders

Multilingual Mix: Example Interpolation Improves Multilingual Neural Machine Translation

Learning Language Specific Sub-network for Multilingual Machine Translation

Multilingual Neural Machine Translation with Language Clustering

Incorporating Language-specific Adapter into Multilingual Neural Machine Translation

Multilingual Agreement for Multilingual Neural Machine Translation

Improving Multilingual Neural Machine Translation by Utilizing Semantic and Linguistic Features

Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder

Improving Language Transfer Capability of Decoder-only Architecture in Multilingual Neural Machine Translation

Variable-length Neural Interlingua Representations for Zero-shot Neural Machine Translation

A Study of Multilingual Neural Machine Translation

Linguistic Knowledge-Aware Neural Machine Translation

Unifying the Convergences in Multilingual Neural Machine Translation

Importance-Based Neuron Selective Distillation for Interference Mitigation in Multilingual Neural Machine Translation

Improving Low-Resource Neural Machine Translation with Weight Sharing

Character-Aware Low-Resource Neural Machine Translation With Weight Sharing And Pre-Training