Abstract:<p>Text classification (TC) is an essential task of natural language processing (NLP). In order to improve the performance of TC, term weighting is often used to obtain effective text representation by assigning appropriate weights to each term. A term weighting scheme is generally composed of term frequency factor, collection frequency factor and normalization factor. The normalization factor is commonly used as an optional factor to offset the influence of document length. Through the investigation of the existing term weighting schemes, we found that most of them focus on finding a more effective collection frequency factor, but rarely pay attention to finding a new term frequency factor. In this paper, we first proposed a new term frequency factor called modified term frequency (MTF). Different from the normalization factor, MTF directly modifies the raw term frequency based on the length information of all training documents. Then we proposed a new term weighting scheme by combining MTF with an existing collection frequency factor called modified distinguishing feature selector (MDFS). We denoted our scheme by MTF-MDFS (MDFS-based MTF). Extensive experimental results on 19 benchmark text datasets and 6 real-world text datasets show that our proposed MTF and MTF-MDFS are all much better than their state-of-the-art competitors in terms of the classification accuracy and the weighted average of <span class="math"><math>F1</math></span> of widely used base classifiers, such as MNB, SVM and LR.</p>

Improve Precategorized Collection Retrieval by Using Supervised Term Weighting Schemes

Supervised Coarse-to-Fine Semantic Hashing for Cross-Media Retrieval.

Exploiting Category Information and Document Information to Improve Term Weighting for Text Categorization

An improved supervised term weighting scheme for text representation and classification

A Novel Term Weighting Scheme for Automated Text Categorization

A LDA Topic Model Based Collection Selection Method for Distributed Information Retrieval

Several alternative term weighting methods for text representation and classification

A study of supervised term weighting scheme for sentiment analysis

Term Selection and Result Reranking for Question Retrieval by Exploiting Hierarchical Classification.

A Study On Feature Weighting In Chinese Text Categorization

A Supervised Term Weighting Scheme for Multi-class Text Categorization

A Comparative Study on Feature Weight in Text Categorization

Modified DFS-based term weighting scheme for text classification

Chinese Text Categorization Based On The Binary Weighting Model With Non-Binary Smoothing

Using modified term frequency to improve term weighting for text classification

DF or IDF? On the Use of Primary Feature Model for Web Information Retrieval

New Feature Selection and Weighting Methods Based on Category Information.

A Term Co-Occurrence Algorithm and the Effect of Co-Occurrence Terms on Result Ranking for Information Retrieval

Reducing Over-Weighting in Supervised Term Weighting for Sentiment Analysis.

Improved Web Image Retrieval by Weighted Image Annotations

A Relevance Evaluation Model of Information Retrieval Based on Weighted Term Frequency