Abstract:Learning bilingual word embeddings can be much easier if the parallel corpora are available with their words well aligned explicitly. However, in most cases, the parallel corpora only provide a set of pairs that are semantically equivalent to each other at sentence level. While algorithms have been proposed to obtain word alignments, good alignments are still hard to achieve. In this study, we propose Bilingual word embeddings with soft alignment (BWESA) to learn bilingual word representations from the parallel corpora without explicit word-level alignment information. At the same time, this method learns to make ‘soft’ alignments between words by approximating a distribution for each word in a sentence to estimate how likely the word is aligned to the words in the parallel translation. Unlike previous methods that typically make use of a predetermined word alignment, our learning strategy makes similar words—properly chosen by the continuously improving word alignment—become closer in the shared vector space during the training process. This study is among the first to learn bilingual word alignments and embeddings in a joint manner. The proposed method was evaluated on two cross-lingual tasks (cross-lingual document classification and word translation) and achieved state-of-the-art or comparable results on all the tasks considered.

Joint Learning of Frequency and Word Embeddings for Multilingual Readability Assessment.

Improve Word Embedding Using Both Writing and Pronunciation.

Enhanced Double-Carrier Word Embedding Via Phonetics and Writing

Mining Coherent Topics in Documents Using Word Embeddings and Large-Scale Text Data

Do Multi-Sense Embeddings Improve Natural Language Understanding?

Enriching Word Embeddings with Domain Knowledge for Readability Assessment.

Joint Learning of Character and Word Embeddings.

Jointly Learning Bilingual Word Embeddings and Alignments

A Graph-based Readability Assessment Method Using Word Coupling.

A Unified Neural Network Model for Readability Assessment with Feature Projection and Length-Balanced Loss

From Word Embedding to Reading Embedding Using Large Language Model, EEG and Eye-tracking

Learning Sense-specific Word Embeddings By Exploiting Bilingual Resources.

Online Learning of Interpretable Word Embeddings

Investigating Language Universal and Specific Properties in Word Embeddings

A Comparison of Word Embeddings for English and Cross-Lingual Chinese Word Sense Disambiguation

Linguistic Features for Readability Assessment

Hierarchical Joint Learning for Chinese Word Embeddings

GRAW+: A Two‐view Graph Propagation Method with Word Coupling for Readability Assessment

Learning Chinese-Japanese Bilingual Word Embedding by Using Common Characters.

A Joint Model for Word Embedding and Word Morphology

LLaVA-Read: Enhancing Reading Ability of Multimodal Language Models