Abstract:Word alignment over parallel corpora has a wide variety of applications, including learning translation lexicons, cross-lingual transfer of language processing tools, and automatic evaluation or analysis of translation outputs. The great majority of past work on word alignment has worked by performing unsupervised learning on parallel texts. Recently, however, other work has demonstrated that pre-trained contextualized word embeddings derived from multilingually trained language models (LMs) prove an attractive alternative, achieving competitive results on the word alignment task even in the absence of explicit training on parallel data. In this paper, we examine methods to marry the two approaches: leveraging pre-trained LMs but fine-tuning them on parallel text with objectives designed to improve alignment quality, and proposing methods to effectively extract alignments from these fine-tuned models. We perform experiments on five language pairs and demonstrate that our model can consistently outperform previous state-of-the-art models of all varieties. In addition, we demonstrate that we are able to train multilingual word aligners that can obtain robust performance on different language pairs. Our aligner, AWESOME (Aligning Word Embedding Spaces of Multilingual Encoders), with pre-trained models is available at https://github.com/neulab/awesome-align

Word Alignment Based on Multi-Grain Model

Word Alignment Combination over Multiple Word Segmentation

Combining Multiple Alignments to Improve Machine Translation.

Research of English-Chinese Alignment at Word Granularity on Parallel Corpora

HybridVocab: Towards Multi-Modal Machine Translation Via Multi-Aspect Alignment

Discriminative Word Alignment over Multiple Word Segmentations

A word alignment model based on multiobjective evolutionary algorithms

Discriminative Word Alignment by Linear Modeling

Diversify and Combine: Improving Word Alignment for Machine Translation on Low-Resource Languages.

Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment

Enhancing Statistical Machine Translation with Character Alignment

A Multilayered Bilingual Word-Alignment Algorithm

Word Alignment by Fine-tuning Embeddings on Parallel Corpora

Improving Statistical Word Alignment with Various Clues.

Towards Integrated Machine Translation Using Structural Alignment From Syntax-Augmented Synchronous Parsing

Improving Function Word Alignment with Frequency and Syntactic Information.

Collocation Extraction Using Monolingual Word Alignment Method.

Transformation from Discontinuous to Continuous Word Alignment Improves Translation Quality

Improving domain-specific word alignment for computer assisted translation

Unsupervised Query Segmentation Using Monolingual Word Alignment Method.

Search for Discriminative Word Alignment via Dual Decomposition