Abstract:Recently, researchers have shown an increasing interest in incorporating linguistic knowledge into neural machine translation (NMT). To this end, previous works choose either to alter the architecture of NMT encoder to incorporate syntactic information into the translation model, or to generalize the embedding layer of the encoder to encode additional linguistic features. The former approach mainly focuses on injecting the syntactic structure of the source sentence into the encoding process, leading to a complicated model that lacks the flexibility to incorporate other types of knowledge. The latter extends word embeddings by considering additional linguistic knowledge as features to enrich the word representation. It thus does not explicitly balance the contribution from word embeddings and the contribution from additional linguistic knowledge. To address these limitations, this paper proposes a knowledge-aware NMT approach that models additional linguistic features in parallel to the word feature. The core idea is that we propose modeling a series of linguistic features at the word level (knowledge block) using a recurrent neural network (RNN). And in sentence level, those word-corresponding feature blocks are further encoded using a RNN encoder. In decoding, we propose a knowledge gate and an attention gate to dynamically control the proportions of information contributing to the generation of target words from different sources. Extensive experiments show that our approach is capable of better accounting for importance of additional linguistic, and we observe significant improvements from 1.0 to 2.3 BLEU points on Chinese$\leftrightarrow$ English and English$\rightarrow$German translation tasks.

Knowledge-Based Semantic Embedding for Machine Translation

English-Chinese Knowledge Base Translation with Neural Network.

Learning Knowledge Graph Embedding in Semantic Space: A Novel Bi-linear Semantic Matching Method

Knowledge-Based Question Answering As Machine Translation.

Linguistic Knowledge-Aware Neural Machine Translation

Evaluation of Machine Translation Based on Semantic Dependencies and Keywords

Better Simultaneous Translation with Monotonic Knowledge Distillation.

Question Answering over Knowledge Base using Language Model Embeddings

Meta-Knowledge Transfer for Inductive Knowledge Graph Embedding

TransMS: Knowledge Graph Embedding for Complex Relations by Multidirectional Semantics

Real-Time Automatic Translation Algorithm for Chinese Subtitles in Media Playback Using Knowledge Base

Knowledge Graph Embedding by Flexible Translation

A Bilingual Graph-Based Semantic Model for Statistical Machine Translation.

Language Models as Knowledge Embeddings

A Knowledge Graph Embedding Method Based on Neural Network

Knowledge Graph Completion Method Combined With Adaptive Enhanced Semantic Information

Revisiting Knowledge Base Embedding as Tensor Decomposition

Improving Multilingual Neural Machine Translation by Utilizing Semantic and Linguistic Features

Automatic Construction of Sememe Knowledge Bases via Dictionaries.

Topic enhanced deep structured semantic models for knowledge base question answering

Continually Learning from Existing Models: Knowledge Accumulation for Neural Machine Translation