Abstract:Syntax has been demonstrated highly effective in neural machine translation (NMT). Previous NMT models integrate syntax by representing 1-best tree outputs from a well-trained parsing system, e.g., the representative Tree-RNN and Tree-Linearization methods, which may suffer from error propagation. In this work, we propose a novel method to integrate source-side syntax implicitly for NMT. The basic idea is to use the intermediate hidden representations of a well-trained end-to-end dependency parser, which are referred to as syntax-aware word representations (SAWRs). Then, we simply concatenate such SAWRs with ordinary word embeddings to enhance basic NMT models. The method can be straightforwardly integrated into the widely-used sequence-to-sequence (Seq2Seq) NMT models. We start with a representative RNN-based Seq2Seq baseline system, and test the effectiveness of our proposed method on two benchmark datasets of the Chinese-English and English-Vietnamese translation tasks, respectively. Experimental results show that the proposed approach is able to bring significant BLEU score improvements on the two datasets compared with the baseline, 1.74 points for Chinese-English translation and 0.80 point for English-Vietnamese translation, respectively. In addition, the approach also outperforms the explicit Tree-RNN and Tree-Linearization methods.

Towards String-to-Tree Neural Machine Translation

Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder

Syntax-Based Context Representation for Statistical Machine Translation.

Inducing Constituency Trees through Neural Machine Translation

Syntax-Aware Complex-Valued Neural Machine Translation

Towards Neural Machine Translation with Latent Tree Attention

Syntax-aware Transformer Encoder for Neural Machine Translation

Syntax-Enhanced Neural Machine Translation with Syntax-Aware Word Representations

Syntax-based Transformer for Neural Machine Translation

Modeling Source Syntax for Neural Machine Translation

Neural machine translation with Gumbel Tree-LSTM based encoder

Neural Machine Translation with Attention Based on a New Syntactic Branch Distance

Incorporating Target Language Semantic Roles into a String-to-tree Translation Model

Predicting Target Language CCG Supertags Improves Neural Machine Translation

Forest-based Tree Sequence to String Translation Model

Language Modeling for Syntax-Based Machine Translation Using Tree Substitution Grammars

Language Modeling for Syntax-Based Machine Translation Using Tree Substitution Grammars: A Case Study on Chinese-English Translation.

Syntax-aware Neural Machine Translation Directed by Syntactic Dependency Degree

Improving Neural Machine Translation with Neural Syntactic Distance

Compositional Neural Machine Translation by Removing the Lexicon from Syntax

Improving Tree-Based Neural Machine Translation with Dynamic Lexicalized Dependency Encoding