Syntax-based word reordering in phrase-based statistical machine translation: why does it work?

S. Zwarts,M. Dras
Abstract:Most natural language applications have some degree of preprocessing of data: tokenisation, stemming and so on. In the domain of Statistical Machine Translation (SMT) it has been shown that word reordering as a preprocessing step can help the translation process, but it is unclear why. We propose two possible reasons for the observed improvement: (1) that the reordering explicitly matches the syntax of the source language more closely to that of the target language; or (2) that it fits the data better to the mechanisms of phrasal SMT. In previous work from German to English, for example, hand-written language-specific reordering rules both match the German more closely to English syntax, and compress heads and dependants into the PSMT phrasal window. Whether the source of the improvement is (1) or (2) has not been determined, although most other work assumes the former. To identify the effects of each possible cause, we carry out two sets of experiments. For (1) we reverse the language-dependent syntactic reordering such that heads and dependants are moved apart. For (2), we propose a generic approach to minimising dependency distances in reordering that does not explicitly match target language word order and that does not require language-specific rules; the aim of which, rather than to beat state-of-the-art systems, is to investigate. The results show that (1) and (2) individually do still lead to improvements in translation quality, but each weaker than the original, suggesting that both features are necessary for a strong improvement. A consequence of this is that is possible to gain half the improvement of language-specific rules through one generic one.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how and to what extent a dynamical system can be represented by a symbolic system. Specifically, the paper explores the theories and methods of representing dynamical systems through so - called Markov partitions. These Markov partitions have special properties, enabling the orbits of the dynamical system to be traced by symbol sequences, thus providing a method of simplifying complex dynamical systems into a form that is easier to analyze. In the paper, particular attention is paid to how to construct such Markov partitions and apply them to specific examples of dynamical systems, such as hyperbolic automorphisms of the two - dimensional torus. Through this method, deep insights into the behavior of the dynamical system can be obtained, including the identification of periodic orbits, almost - periodic orbits and dense orbits, etc. In addition, the paper also discusses how to construct Markov partitions in reverse from the symbolic representation, which involves an interesting question: should Markov partitions be constructed first to obtain the symbolic representation, or should the symbolic representation be generated first to obtain the Markov partition? The answer to this question is crucial for understanding the relationship between dynamical systems and symbolic dynamics.