Abstract:It is acknowledged that neural machine translation(NMT) can be improved by considering context information. Nevertheless, the progress of context-aware NMT has encountered some challenges. Firstly, effectively utilizing valuable information contained in context is still challenging. Moreover, as the number of sentences increases, the parameters of context-aware NMT models will surge, which costs computing power and prevents them from transferring to other translation tasks. Therefore, we propose a hierarchical shallow attention structure for document-level NMT to tackle the problems above. We employ hierarchical encoders to extract both sentence and context information. Then the integration of hierarchical attention is incorporated with the self-attention of the target sentences in the decoding phase. Moreover, we employ shallow attention to reduce model complexity. Assessments of several linguistic phenomena demonstrate that the proposed approach can balance model complexity and translation performance while getting SOTA BLEU scores in several translation tasks.

An Orientation Model for Hierarchical Phrase-Based Translation

Lexical Reordering for Hierarchical Phrase-based Translation

A Lexicalized Reordering Model for Hierarchical Phrase-based Translation

A Head-Annotated Synchronous Context-Free Grammar for Hierarchical Phrase-Based Translation

Head-modifier relation based non-lexical reordering model for phrase-based translation

A Unified and Discriminative Soft Syntactic Constraint Model for Hierarchical Phrase-based Translation.

An Improved Hierarchical Phrase Based Machine Translation Model

Two-Neighbor Orientation Model with Cross-Boundary Global Contexts.

Tree-State Based Rule Selection Models for Hierarchical Phrase-Based Machine Translation.

Soft Syntactic Constraints for Hierarchical Phrase-Based Translation Using Latent Syntactic Distributions.

A Joint Rule Selection Model for Hierarchical Phrase-Based Translation.

Hierarchical Phrase Table Combination for Machine Translation.

Learning Semantic Representations for Nonterminals in Hierarchical Phrase-Based Translation.

Word-Level Reordering Model for Phrase-Based SMT

Comparative Study on Hierarchical Phrase Structures and Linguistic Phrase Structures.

Using Features from Topic Models to Alleviate Over-Generation in Hierarchical Phrase-Based Translation

A Topic Similarity Model for Hierarchical Phrase-based Translation.

Harmonizing Word Alignments and Syntactic Structures for Extracting Phrasal Translation Equivalents

A Tree Sequence Alignment-based Tree-to-Tree Translation Model.

Learning Hierarchical Translation Spans.

Context Modeling with Hierarchical Shallow Attention Structure for Document-Level NMT