Abstract:Word alignment over parallel corpora has a wide variety of applications, including learning translation lexicons, cross-lingual transfer of language processing tools, and automatic evaluation or analysis of translation outputs. The great majority of past work on word alignment has worked by performing unsupervised learning on parallel texts. Recently, however, other work has demonstrated that pre-trained contextualized word embeddings derived from multilingually trained language models (LMs) prove an attractive alternative, achieving competitive results on the word alignment task even in the absence of explicit training on parallel data. In this paper, we examine methods to marry the two approaches: leveraging pre-trained LMs but fine-tuning them on parallel text with objectives designed to improve alignment quality, and proposing methods to effectively extract alignments from these fine-tuned models. We perform experiments on five language pairs and demonstrate that our model can consistently outperform previous state-of-the-art models of all varieties. In addition, we demonstrate that we are able to train multilingual word aligners that can obtain robust performance on different language pairs. Our aligner, AWESOME (Aligning Word Embedding Spaces of Multilingual Encoders), with pre-trained models is available at https://github.com/neulab/awesome-align

Search for Discriminative Word Alignment via Dual Decomposition

Search for Discriminative Word Alignment via Dual Decomposition

Discriminative Word Alignment by Linear Modeling

Discriminative Word Alignment over Multiple Word Segmentations

Joint Word Alignment and Bilingual Named Entity Recognition Using Dual Decomposition.

Word Alignment by Fine-tuning Embeddings on Parallel Corpora

Alignment-Aware Word Distance.

BinaryAlign: Word Alignment as Binary Sequence Labeling

Consistency-Aware Search for Word Alignment.

Combining Multiple Alignments to Improve Machine Translation.

Research of English-Chinese Alignment at Word Granularity on Parallel Corpora

Word Alignment Combination over Multiple Word Segmentation

Diversify and Combine: Improving Word Alignment for Machine Translation on Low-Resource Languages.

A Beam Search Algorithm for ITG Word Alignment.

Optimizing Word Alignment Combination for Phrase Table Training

Improving Word Alignment by Semi-Supervised Ensemble.

Word Alignment in the Era of Deep Learning: A Tutorial

Improving Statistical Word Alignment with Various Clues.

Integration Algorithm Of English-Chinese Word Segmentation And Alignment

Towards Better Word Alignment in Transformer.

: Improving Domain-Specific Word Alignment with a General Bilingual Corpus