Abstract:Multilingual neural machine translation aims to translate multiple language pairs in a single model and has shown great success thanks to the knowledge transfer across languages with the shared parameters. Despite promising, this share-all paradigm suffers from insufficient ability to capture language-specific features. Currently, the common practice is to insert or search language-specific networks to balance the shared and specific features. However, those two types of features are not sufficient enough to model the complex commonality and divergence across languages, such as the locally shared features among similar languages, which leads to sub-optimal transfer, especially in massively multilingual translation. In this paper, we propose a novel token-level feature mixing method that enables the model to capture different features and dynamically determine the feature sharing across languages. Based on the observation that the tokens in the multilingual model are usually shared by different languages, we we insert a feature mixing layer into each Transformer sublayer and model each token representation as a mix of different features, with a proportion indicating its feature preference. In this way, we can perform fine-grained feature sharing and achieve better multilingual transfer. Experimental results on multilingual datasets show that our method outperforms various strong baselines and can be extended to zero-shot translation. Further analyses reveal that our method can capture different linguistic features and bridge the representation gap across languages.

Joint Tokenization and Translation

Joint tokenization, parsing, and translation

Joint Decoding with Multiple Translation Models.

Joint Parsing and Translation

Joint Training and Decoding for Multilingual End-to-End Simultaneous Speech Translation

Better Simultaneous Translation with Monotonic Knowledge Distillation.

Joint Training and Decoding Using Virtual Nodes for Cascaded Segmentation and Tagging Tasks.

Joint decoding of multi-confusion-network in MT system combination

Joint bilingual name tagging for parallel corpora

PredToken: Predicting Unknown Tokens and Beyond with Coarse-to-Fine Iterative Decoding

Synchronous Speech Recognition and Speech-to-Text Translation with Interactive Decoding.

Adaptive Token-level Cross-lingual Feature Mixing for Multilingual Neural Machine Translation

Joint Training for Pivot-based Neural Machine Translation.

Token-Level Serialized Output Training for Joint Streaming ASR and ST Leveraging Textual Alignments

Joint Decoding of Tandem and Hybrid Systems for Improved Keyword Spotting on Low Resource Languages

Enhanced Neural Machine Translation by Joint Decoding with Word and POS-tagging Sequences.

Sequence Generation with Mixed Representations.

Joint Word Alignment and Bilingual Named Entity Recognition Using Dual Decomposition.

Lattice-based System Combination for Statistical Machine Translation.

Joint Decoding of CTC Based Systems for Speech Recognition

Flexible and Efficient Hypergraph Interactions for Joint Hierarchical and Forest-to-String Decoding.