Abstract:Maximum Entropy (MaxEnt) model has been proven to be a very effective approach in the topic classification task, where a specific topic from a pre-defined topic set will be assigned to each sentence. Although it is originally developed based on the motivation of maximizing the conditional probability entropy under certain constraints, MaxEnt model is indeed an exponential distribution model that maximizes the log-likelihood of the training data. This log-likelihood criterion bears similarity with the classification accuracy criterion, which is the ultimate performance measure of a topic classifier. But these two criterion still differ from each other, and their discrepancy consequently reduces the benefit of optimization in improving classification accuracy. In this paper we propose to use different objective functions, which are closer to the classification accuracy criterion, to replace the log-likelihood objective used in the MaxEnt model estimation process. Specifically, we propose a Summation-Log K-norm objective and a Summation K-norm objective. Our experiments conducted on two large volume topic classification dataset prove the effectiveness of our new objectives in improving topic classification performance on top of the state-of-art MaxEnt model.

An Improved Feature Representation Method for Maximum Entropy Model.

Analyzing the Incomplete Data Based on the Improved Maximum Entropy Model Analyzing the Incomplete Data Based on the Improved Maximum Entropy Model

Improving Feature Extraction in Named Entity Recognition Based on Maximum Entropy Model

An Improved Maximum Entropy Language Model and Its Application

Improved Topic Classification over Maximum Entropy Model Using K-Norm Based New Objectives

Enhancing Web Text Clustering Accuracy and Efficiency With a Maximum Entropy Function Model: Overcoming High-Dimensional and Directional Challenges

Propagating Image-Level Part Statistics to Enhance Object Detection

Combined maximum entropy language model using different feature sets

Fuzzy Entropy Based Max-Relevancy and Min-Redundancy Feature Selection.

Efficient representation and fast look-up of Maximum Entropy language models.

Improving the Accuracy of Principal Component Analysis by the Maximum Entropy Method

Exploiting Pinyin Constraints in Pinyin-to-Character Conversion Task: a Class-Based Maximum Entropy Markov Model Approach

A Feature Matching Method: Sparse Feature Tree

Hidden Maximum Entropy Approach for Visual Concept Modeling

A Maximum Entropy Markov Model for Chunking

Probabilistic Outputs for Support Vector Machines Based on the Maximum Entropy Estimation

Finding The Optimal Feature Representations For Bayesian Network Learning

Principal Component Analysis Based on Non-Parametric Maximum Entropy

Supervised Feature Extraction Algorithm Based on Improved Polynomial Entropy

Optimization Feature Compression and Fnn Realization

Robust Discriminant Analysis Based on Nonparametric Maximum Entropy