Abstract:Bilingual word embedding has been shown to be helpful for Statistical Machine Translation (SMT). However, most existing methods suffer from two obvious drawbacks. First, they only focus on simple contexts such as an entire document or a fixed-sized sliding window to build word embedding and ignore latent useful information from the selected context. Second, the word sense but not the word should be the minimal semantic unit; however, most existing methods still use word representation. To overcome these drawbacks, this article presents a novel Graph-Based Bilingual Word Embedding (GBWE) method that projects bilingual word senses into a multidimensional semantic space. First, a bilingual word co-occurrence graph is constructed using the co-occurrence and pointwise mutual information between the words. Then, maximum complete subgraphs (cliques), which play the role of a minimal unit for bilingual sense representation, are dynamically extracted according to the contextual information. Consequently, correspondence analysis, principal component analyses, and neural networks are used to summarize the clique-word matrix into lower dimensions to build the embedding model. Without contextual information, the proposed GBWE can be applied to lexical translation. In addition, given contextual information, GBWE is able to give a dynamic solution for bilingual word representations, which can be applied to phrase translation and generation. Empirical results show that GBWE can enhance the performance of lexical translation, as well as Chinese/French-to-English and Chinese-to-Japanese phrase-based SMT tasks (IWSLT, NTCIR, NIST, and WAT).

Learning Bilingual Distributed Phrase Represenations for Statistical Machine Translation

Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

Learning Semantic Representations for Nonterminals in Hierarchical Phrase-Based Translation.

Phrase-Based & Neural Unsupervised Machine Translation

Graph-Based Bilingual Word Embedding for Statistical Machine Translation

Non-linear Learning for Statistical Machine Translation.

Translating Phrases in Neural Machine Translation

A Bilingual Graph-Based Semantic Model for Statistical Machine Translation.

Connecting Phrase Based Statistical Machine Translation Adaptation.

DRWS: A Model for Learning Distributed Representations for Words and Sentences.

Improve Statistical Machine Translation with Context-Sensitive Bilingual Semantic Embedding Model.

A Topic Similarity Model for Hierarchical Phrase-based Translation.

Bilingual Correspondence Recursive Autoencoder for Statistical Machine Translation

Learning Bilingual Sentence Representations for Quality Estimation of Machine Translation.

"The Sum of Its Parts": Joint Learning of Word and Phrase Representations with Autoencoders

Incorporating Linguistic Knowledge for Learning Distributed Word Representations.

Learning local word reorderings for hierarchical phrase-based statistical machine translation

Learning Distributed Word Representations for Bidirectional LSTM Recurrent Neural Network

Syntax-Based Context Representation for Statistical Machine Translation.

Learning Distributed Representations of Sentences from Unlabelled Data

Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine Translation