Abstract:In Text Categorization (TC) based on the vector space model, feature weighting is vital for the categorization effectiveness. Various non-binary weighting schemes are widely used for this purpose. By emphasizing the category discrimination capability of features, the paper firstly puts forward a new weighting scheme TF*IDF*IG. Upon the fact that refined statistics may have more chance to meet sparse data problem, we re-evaluate the role of the Binary Weighting Model (BWM) in TC for further consideration. As a consequence, a novel approach named the Binary Weighting Model with Non-Binary Smoothing (BWM-NBS) is then proposed so as to overcome the drawback of BWM. A TC system for Chinese texts using words as features is implemented. Experiments on a large-scale Chinese document collection with 71,674 texts show that the F1 metric of categorization performance of BWM-NBS gets to 94.9% in the best case, which is 26.4% higher than that of TF*IDF, 19.1% higher than that of TF*IDF*IG, and 5.8% higher than that of BWM under the same condition. Moreover, BWM-NBS exhibits the strong stability in categorization performance.

An Improved Bayesian Text Categorization System

A Technique For Improving The Performance Of Naive Bayes Text Classification

Way of text classification based on Bayes

The Bayesian Text Categorization Based on Extraction of Effectual Features

Improving Text Categorization Using the Importance of Words in Different Categories

Text Categorization with Lee Model

Naive bayes text categorization using improved feature selection

Text Categorization Method Based on Improved Mutual Information and Characteristic Weights Evaluation Algorithms

An Enhanced Text Categorization Method Based on Improved Text Frequency Approach and Mutual Information Algorithm

An Approach to Automatic Text Categorization Based on Combination of Rule and Bayesian Method

Chinese Short-Text Categorization Based on the Key Classification Dictionary Words

Chinese Text Categorization Based On The Binary Weighting Model With Non-Binary Smoothing

Text Classification Based on a Novel Bayesian Hierarchical Model

Patent Categorization Based on Bayes Model

Based on a Kind of Method to Categorize Text of Improve the Mutual Information

Text Classification Combined an Improved CHI and Category Relevance Factor

Improved KNN Text Categorization

An Improved Random Forest Classifier for Text Categorization.

A Study On Feature Weighting In Chinese Text Categorization

Context Based Feature Description Model in Chinese Text Categorization

Classifying Chinese Texts in Two Steps.