Abstract:Neural network language models (LMs) are shown to be effective in improving the performance of statistical machine translation (SMT) systems. However, state-of-the-art neural network LMs usually use words before the current position as context and neglect global topic information, which can helpmachine translation (MT) systems to select better translation candidates from a higher perspective. In this work, we propose improvement of the state-of-the-art feedforward neural language model with topic information. Two main issues need to be tackled when adding topics into neural network LMs for SMT: one is how to incorporate topics to the neural network; the other is how to get target-side topic distribution before translation. We incorporate topics by appending topic distribution to the input layer of a feedforward LM. We adopt a multinomial logistic-regression (MLR) model to predict the target-side topic distribution based on source side information. Moreover, we propose a feedforward neural network model to learn joint representations on the source side for topic prediction. LM experiments demonstrate that the perplexity on validation set can be greatly reduced by the topic-enhanced feedforward LM, and the prediction of target-side topics can be improved dramatically with the MLR model equipped with the joint source representations. A final MT experiment, conducted on a large-scale Chinese-English dataset, shows that our feedforward LM with predicted topics improves the translation performance against a strong baseline.

A New Topic-Based Language Model Adaptation

Agmma: A Novel Incremental Adaptation Method And Its Application To Speaker Recognition

Language model adaptation based on correction information for interactive speech transcription

Speaker adaptation based on combination of MAP estimation and weighted neighbor regression

An Online Incremental Language Model Adaptation Method

Speaker Adaptation with MAP Estimation and Weighted Neighbor Regression

Refine Bigram PLSA Model by Assigning Latent Topics Unevenly

Translation Model Adaptation for Statistical Machine Translation with Monolingual Topic Information.

Linguistic tree based maximum likelihood model interpolation

Adaptation of Language Models for SMT Using Neural Networks with Topic Information.

A New Subspace Based Speaker Adaptation Method

Speaker adaptation using maximum likelihood model interpolation

MAP-based Speaker Adaptation in Speech Synthesis

Codebook-Based Speaker Adaptation

A new latent semantic analysis language model

Language Model Adaptation Based on the Classification of a Trigram's Language Style Feature

An Improved Cross-Language Model Adaptation Method for Speech Synthesis

Model Adaptation Using the Projection to Latent Structure Algorithm

Interpolation adaptation algorithm based on gaussian similarity analysis

Smoothing Algorithm of the Task Adaptation Chinese N-gram Model

A Smoothing Algorithm For The Task Adaptation Chinese Trigram Model