Abstract:Word alignment which aims to extract lexicon translation equivalents between source and target sentences, serves as a fundamental tool for natural language processing. Recent studies in this area have yielded substantial improvements by generating alignments from contextualized embeddings of the pre-trained multilingual language models. However, we find that the existing approaches capture few interactions between the input sentence pairs, which degrades the word alignment quality severely, especially for the ambiguous words in the monolingual context. To remedy this problem, we propose Cross-Align to model deep interactions between the input sentence pairs, in which the source and target sentences are encoded separately with the shared self-attention modules in the shallow layers, while cross-lingual interactions are explicitly constructed by the cross-attention modules in the upper layers. Besides, to train our model effectively, we propose a two-stage training framework, where the model is trained with a simple Translation Language Modeling (TLM) objective in the first stage and then finetuned with a self-supervised alignment objective in the second stage. Experiments show that the proposed Cross-Align achieves the state-of-the-art (SOTA) performance on four out of five language pairs.

Combining Multiple Alignments to Improve Machine Translation.

Diversify and Combine: Improving Word Alignment for Machine Translation on Low-Resource Languages.

Combining Many Alignments for Speech to Speech Translation

Word Alignment Combination over Multiple Word Segmentation

Improving Alignment of System Combination by Using Multi-objective Optimization.

Improve The Statistical Machine Translation Performance By Refining The Word Alignments

A Comparative Study of Hypothesis Alignment and its Improvement for Machine Translation System Combination.

Experiments for various alignment models in Chinese-to-English SMT

Optimizing Word Alignment Combination for Phrase Table Training

Transformation from Discontinuous to Continuous Word Alignment Improves Translation Quality

Enhancing Statistical Machine Translation with Character Alignment

Towards Better Word Alignment in Transformer.

Improving Statistical Word Alignment with Various Clues.

Weighted Alignment Matrices for Statistical Machine Translation.

Realignment from Finer-grained Alignment to Coarser-grained Alignment to Enhance Mongolian-Chinese SMT.

On The Alignment Problem In Multi-Head Attention-Based Neural Machine Translation

Improving Translation Memory with Word Alignment Information

Consistency-Aware Search for Word Alignment.

Word Realignment for Statistical Machine Translation

Sub-Word Alignment is Still Useful: A Vest-Pocket Method for Enhancing Low-Resource Machine Translation

Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment