Abstract:Multimodal Machine Translation (MMT) aims to enhance translation quality by incorporating information from other modalities (usually images). However, dominant MMT models do not consider that visual features not only provide supplementary information also introduce much noise. In this paper, we propose the visual features filter to solve this issue. Specifically, we adopt a soft-lookup function to select the visual features relevant to the text and then use these visual features as pseudo-words concatenating with a text representation. In addition, our model conducts two-pass decoding. The secondarypass decoding amounts to polishing which can identify errors in draft translations. The reason is that polishing expands the view in the process of decoding each target token, providing more contextual information. Besides, since most words in draft translations can be copied to final translations, we further equip our model with the copying mechanism to reserve those words that do not need to be corrected. MMT has achieved success in some mainstream languages at present. In order to promote the development of MMT in low-resource languages such as Mongolian, we deploy our model to the Mongolian→Chinese translation task. We expand Multi30k dataset to synthetic Mongolian and Chinese descriptions. Experiments on synthetic Mongolian and Chinese datasets demonstrate that our model can bring significant improvements.

Combining Discrete Lexicon Probabilities with NMT for Low-Resource Mongolian-Chinese Translation

Adapting Attention-based Neural Network to Low-resource Mongolian-Chinese Machine Translation

Low-Resource Neural Machine Translation Using XLNet Pre-training Model

Sub-word Embedding Auxiliary Encoding in Mongolian-Chinese Neural Machine Translation

Neural System Combination For Machine Translation

Character-Aware Low-Resource Neural Machine Translation With Weight Sharing And Pre-Training

Key Research of Pre-Processing on Mongolian-Chinese Neural Machine Translation

Syntactic Relevance XLNet Word Embedding Generation in Low-Resource Machine Translation

Exploiting Japanese–Chinese Cognates with Shared Private Representations for NMT

Mongolian-Chinese Unsupervised Neural Machine Translation With Lexical Feature

Multimodal Neural Machine Translation for Mongolian to Chinese

Towards Integrated Classification Lexicon for Handling Unknown Words in Chinese-Vietnamese Neural Machine Translation

Improving Low-Resource Neural Machine Translation with Weight Sharing

Improving Performance of NMT Using Semantic Concept of WordNet Synset

Enhanced Neural Machine Translation by Joint Decoding with Word and POS-tagging Sequences.

Mongolian-Chinese Neural Machine Translation with Priori Information

Higher Target Relevance Parallel Machine Translation with Low-Frequency Word Enhancement

Bridging Neural Machine Translation and Bilingual Dictionaries

Semantic Perception-Oriented Low-Resource Neural Machine Translation

An Unknown Word Processing Method in NMT by Integrating Syntactic Structure and Semantic Concept.

Research on Mongolian-Chinese Translation Model Based on Transformer with Soft Context Data Augmentation Technique