Improving Tree-Based Neural Machine Translation with Dynamic Lexicalized Dependency Encoding

Baosong Yang,Derek F. Wong,Lidia S. Chao,Min Zhang
DOI: https://doi.org/10.1016/j.knosys.2019.105042
IF: 8.139
2020-01-01
Knowledge-Based Systems
Abstract:Tree-to-sequence neural machine translation models have proven to be effective in learning the semantic representations from the exploited syntactic structure. Despite their success, tree-to-sequence models have two major issues: (1) the embeddings of constituents at the higher tree levels tend to contribute less in translation; and (2) using a single set of model parameters is difficult to fully capture the syntactic and semantic richness of linguistic phrases. To address the first problem, we proposed a lexicalized dependency model, in which the source-side lexical representations are learned in a head-dependent fashion following a dependency graph. Since the number of dependents is variable, we proposed a variant recurrent neural network (RNN) to jointly consider the long-distance dependencies and the sequential information of words. Concerning the second problem, we adopt a latent vector to dynamically condition the parameters for the composition of each node representation. Experimental results reveal that the proposed model significantly outperforms the recently proposed tree-based methods in English-Chinese and English-German translation tasks with even far fewer parameters. (C) 2019 Elsevier B.V. All rights reserved.
What problem does this paper attempt to address?