Abstract:An adaptive statistical language model is described, which successfully integrates long distance linguistic information with other knowledge sources. Most existing statistical language models exploit only the immediate history of a text. To extract information from further back in the document's history, we propose and usetrigger pairsas the basic information bearing elements. This allows the model to adapt its expectations to the topic of discourse. Next, statistical evidence from multiple sources must be combined. Traditionally, linear interpolation and its variants have been used, but these are shown here to be seriously deficient. Instead, we apply the principle of Maximum Entropy (ME). Each information source gives rise to a set of constraints, to be imposed on the combined estimate. The intersection of these constraints is the set of probability functions which are consistent with all the information sources. The function with the highest entropy within that set is the ME solution. Given consistent statistical evidence, a unique ME solution is guaranteed to exist, and an iterative algorithm exists which is guaranteed to converge to it. The ME framework is extremely general: any phenomenon that can be described in terms of statistics of the text can be readily incorporated. An adaptive language model based on the ME approach was trained on theWall Street Journalcorpus, and showed a 32–39% perplexity reduction over the baseline. When interfaced to SPHINX-II, Carnegie Mellon's speech recognizer, it reduced its error rate by 10–14%. This thus illustrates the feasibility of incorporating many diverse knowledge sources in a single, unified statistical framework.

News-Oriented Keyword Indexing with Maximum Entropy Principle.

Research on Maximum Entropy Model for Keyword Indexing

News-oriented Automatic Chinese Keyword Indexing

Automatic image annotation using maximum entropy model

Application of Maximum Entropy Method in Chinese-English Cross Language Information Retrieval

Empirical Analysis on a Keyword-Based Semantic System

A Maximum Entropy Model for Automatic Summarization

An Improved Maximum Entropy Language Model and Its Application

A maximum entropy approach to adaptive statistical language modelling

Keyword Extraction Based on Tf/idf for Chinese News Document

Research on Me-based Chinese NER Model

Semantic Chunk Annotation for questions using Maximum Entropy

Improved Topic Classification over Maximum Entropy Model Using K-Norm Based New Objectives

Single Document Keyword Extraction for Internet News Articles

Keyword Extraction Approach Based on Probabilistic-Entropy, Graph, and Neural Network Methods

News keyword extraction algorithm based on semantic clustering and word graph model

Improving Feature Extraction in Named Entity Recognition Based on Maximum Entropy Model

A Maximum Entropy Approach to Chinese Spelling Check.

A Combining Approach to Automatic Keyphrases Indexing for Chinese News Documents

Enhancing Web Text Clustering Accuracy and Efficiency With a Maximum Entropy Function Model: Overcoming High-Dimensional and Directional Challenges

Zero-Anaphora Resolution in Chinese Using Maximum Entropy