Abstract:Neural network language models (LMs) are shown to be effective in improving the performance of statistical machine translation (SMT) systems. However, state-of-the-art neural network LMs usually use words before the current position as context and neglect global topic information, which can helpmachine translation (MT) systems to select better translation candidates from a higher perspective. In this work, we propose improvement of the state-of-the-art feedforward neural language model with topic information. Two main issues need to be tackled when adding topics into neural network LMs for SMT: one is how to incorporate topics to the neural network; the other is how to get target-side topic distribution before translation. We incorporate topics by appending topic distribution to the input layer of a feedforward LM. We adopt a multinomial logistic-regression (MLR) model to predict the target-side topic distribution based on source side information. Moreover, we propose a feedforward neural network model to learn joint representations on the source side for topic prediction. LM experiments demonstrate that the perplexity on validation set can be greatly reduced by the topic-enhanced feedforward LM, and the prediction of target-side topics can be improved dramatically with the MLR model equipped with the joint source representations. A final MT experiment, conducted on a large-scale Chinese-English dataset, shows that our feedforward LM with predicted topics improves the translation performance against a strong baseline.

A Topic-Triggered Language Model for Statistical Machine Translation.

A Context-Aware Topic Model for Statistical Machine Translation.

Adaptation of Language Models for SMT Using Neural Networks with Topic Information.

Translation Model Adaptation for Statistical Machine Translation with Monolingual Topic Information.

A Joint Topical N-Gram Language Model Based on LDA

Tipster: A Topic-Guided Language Model for Topic-Aware Text Segmentation.

A trigger language model-based IR system

Prompting Large Language Models for Topic Modeling

Topical Word Trigger Model for Keyphrase Extraction.

A New Topic-Based Language Model Adaptation

Revisiting Topic-Guided Language Models

LTSG: Latent Topical Skip-Gram for Mutually Learning Topic Model and Vector Representations

Trigger-based language models: a maximum entropy approach

Language Model is a Branch Predictor for Simultaneous Machine Translation

TOPICS TRANSLATION MODEL-BASED BILINGUAL TEXT ERRORS CORRECTION

Prevent the Language Model from being Overconfident in Neural Machine Translation

Language Model-Driven Unsupervised Neural Machine Translation

Learning Topical Translation Model for Microblog Hashtag Suggestion

Topic-based mixture language modelling

Neural Machine Translation Advised by Statistical Machine Translation

Neural Machine Translation Model with a Large Vocabulary Selected by Branching Entropy