Abstract:Even though a linguistics-free sequence to sequence model in neural machine translation (NMT) has certain capability of implicitly learning syntactic information of source sentences, this paper shows that source syntax can be explicitly incorporated into NMT effectively to provide further improvements. Specifically, we linearize parse trees of source sentences to obtain structural label sequences. On the basis, we propose three different sorts of encoders to incorporate source syntax into NMT: 1) Parallel RNN encoder that learns word and label annotation vectors parallelly; 2) Hierarchical RNN encoder that learns word and label annotation vectors in a two-level hierarchy; and 3) Mixed RNN encoder that stitchingly learns word and label annotation vectors over sequences where words and labels are mixed. Experimentation on Chinese-to-English translation demonstrates that all the three proposed syntactic encoders are able to improve translation accuracy. It is interesting to note that the simplest RNN encoder, i.e., Mixed RNN encoder yields the best performance with an significant improvement of 1.4 BLEU points. Moreover, an in-depth analysis from several perspectives is provided to reveal how source syntax benefits NMT.

Statistical Translation Model Based On Source Syntax Structure

Comparative Study of Statistical Translation Models on Chinese-English Speech Translation

A Source Dependency Model for Statistical Machine Translation.

Forest-based Tree Sequence to String Translation Model

A Tree-to-tree Alignment-Based Model for Statistical Machine Translation

Meta-Structure Transformation Model for Statistical Machine Translation.

Improving Phrase-Based Statistical Machine Translation Models by Incorporating Syntax-Based Language Models

Language Modeling for Syntax-Based Machine Translation Using Tree Substitution Grammars

Simple but Effective Approaches to Improving Tree-to-tree Model

Language Modeling for Syntax-Based Machine Translation Using Tree Substitution Grammars: A Case Study on Chinese-English Translation.

Syntax Encapsulated Phrase Model for Statistical Machine Translation

Syntax-Based Chinese-Vietnamese Tree-to-Tree Statistical Machine Translation with Bilingual Features

A Study of Translation Rule Classification for Syntax-Based Statistical Machine Translation

A Dependency Edge-based Transfer Model for Statistical Machine Translation.

Some Issues of Syntax-based Statistical Machine Translation

A Structured Language Model for Incremental Tree-to-String Translation.

A Tree Sequence Alignment-based Tree-to-Tree Translation Model.

A Statistical Machine Translation Model Based on a Synthetic Synchronous Grammar.

A Dependency Treelet String Correspondence Model for Statistical Machine Translation.

Modeling Source Syntax for Neural Machine Translation

Dependency-based bracketing transduction grammar for statistical machine translation