Abstract:Abstract Term weighting is essential for text classification tasks, and thus various supervised term-weighting (STW) methods have been designed and presented in recent years, such as TF (term frequency)-IG (information gain), TF-MI (mutual information), TF-RF (relevance frequency), and TF-IDF (inverse document frequency)-ICSDF (inverse class space density frequency). Unlike other schemes, TF-IDF-ICSDF considers not only the local factor (i.e. TF) and the category factor (i.e. ICSDF) but also the global factor (i.e. IDF) in the weighting process. Hence, a natural question is whether IDF is really useful for improving the classification performance of STW schemes. To explore this issue, a generic multi-level framework composed of term-level, text-level, and category-level is first established, which corresponds to local factor, global factor, and category factor, respectively. Based on the generic multi-level framework, a new two-level STW method, TF-ICSDF, can be generated by removing the IDF from the TF-IDF-ICSDF scheme. Conversely, we also integrated the IDF with other two-level STW schemes (e.g. TF-IG, TF-MI, TF-RF) to obtain several three-level STW schemes. We verified the general classification performance of our proposed STW schemes on three open benchmark datasets. The results manifest that performance can usually be boosted if IDF is incorporated into the STW schemes, indicating that weighting terms utilizing the IDF factor could provide better text representation. Therefore, the generic multi-level framework and STW schemes we proposed are effective.

DF or IDF? on the Use of HTML Primary Feature Fields for Web IR.

DF or IDF? On the Use of Primary Feature Model for Web Information Retrieval

Data Warehouse Native Feature Based OLAP Querying with Keywords

Exploiting Category Information and Document Information to Improve Term Weighting for Text Categorization

DIFP: A candidate facet generation optimization method towards keyword-driven analytical processing

A Learning-Based Term-Weighting Approach for Information Retrieval

Feature Weighting for Improving Document Image Retrieval System Performance

A Term Co-Occurrence Algorithm and the Effect of Co-Occurrence Terms on Result Ranking for Information Retrieval

A generic multi-level framework for building term-weighting schemes in text classification

A Relevance Evaluation Model of Information Retrieval Based on Weighted Term Frequency

An Integrated Search Framework for Leveraging the Knowledge-Based Web Ecosystem

Improve Precategorized Collection Retrieval by Using Supervised Term Weighting Schemes

The Use of Dependency Relation Graph to Enhance the Term Weighting in Question Retrieval.

Class-indexing-based term weighting for automatic text classification

A Novel Term Weighing Scheme Towards Efficient Crawl of Textual Databases

LAF: a New XML Encoding and Indexing Strategy for Keyword‐based XML Search

Modified DFS-based term weighting scheme for text classification

DF-Miner: Domain-specific Facet Mining by Leveraging the Hyperlink Structure of Wikipedia

Several alternative term weighting methods for text representation and classification

Using modified term frequency to improve term weighting for text classification

A Novel Term Weighting Scheme for Automated Text Categorization