Abstract:Recently many new techniques have been proposed for language modeling, such as ME, MEMM and CRF. However, the ngram model is still a staple in practical applications. It is well worthy of studying how to improve the performance of the ngram model. This paper enhances the traditional ngram model by relaxing the stationary hypothesis on the Markov chain and exploiting the word positional information. Such an assumption is made that the probability of the current word is determined not only by history words but also by the words positions in the sentence. The non-stationary ngram model (NS ngram model) is proposed. Several related issues are discussed in detail, including the definition of the NS ngram model, the representation of the word positional information and the estimation of the conditional probability. In addition, three smoothing approaches are proposed to solve the data sparseness problem of the NS ngram model. Several smoothing algorithms are presented in each approach. In the experiments, the NS ngram model is evaluated on the pinyin-to-character conversion task which is the core technique of the Chinese text input method. Experimental results show that the NS ngram model outperforms the traditional ngram model significantly by the exploitation of the word positional information. In addition, the proposed smoothing techniques solve the data sparseness problem of the NS ngram model effectively with great error rate reduction.

A Smoothing Algorithm For The Task Adaptation Chinese Trigram Model

Smoothing Algorithm of the Task Adaptation Chinese N-gram Model

Language Model Adaptation Based on the Classification of a Trigram's Language Style Feature

Agmma: A Novel Incremental Adaptation Method And Its Application To Speaker Recognition

A Language Model Adaptation Approach Based on Text Classification.

Discriminative Language Model Adaptation for Mandarin Broadcast Speech Transcription and Translation

An Empirical Study of Non-Stationary Ngram Model and its Smoothing Techniques

A similarity-based smoothing algorithm for Chinese language modeling and its application in pinyin-to-character conversion

A New Topic-Based Language Model Adaptation

An Active Learning Approach to Task Adaptation.

An Adaptive Post-processing Method using Proofreading Information for Chinese Character Recognition

Toward a Unified Approach to Statistical Language Modeling for Chinese

A Word Language Model Based Contextual Language Processing On Chinese Character Recognition

Chinese-English Smt for Cross-Language Dialogue Agent Support

Language model adaptation based on correction information for interactive speech transcription

Attention-Guided Adaptation for Code-Switching Speech Recognition

Rethinking LLM Language Adaptation: A Case Study on Chinese Mixtral

Human-Computer Interactive Chinese Word Segmentation: an Adaptive Dirichlet Process Mixture Model Approach.

An Improved Cross-Language Model Adaptation Method for Speech Synthesis

Cluster Adaptive Training for Deep Neural Network

CTC Regularized Model Adaptation for Improving LSTM RNN Based Multi-Accent Mandarin Speech Recognition