Abstract:In vector space model (VSM), text representation is the task of transforming the content of a textual document into a vector in the term space so that the document could be recognized and classified by a computer or a classifier. Different terms (i.e. words, phrases, or any other indexing units used to identify the contents of a text) have different importance in a text. The term weighting methods assign appropriate weights to the terms to improve the performance of text categorization. In this study, we investigate several widely-used unsupervised (traditional) and supervised term weighting methods on benchmark data collections in combination with SVM and kNN algorithms. In consideration of the distribution of relevant documents in the collection, we propose a new simple supervised term weighting method, i.e. tf.rf, to improve the terms' discriminating power for text categorization task. From the controlled experimental results, these supervised term weighting methods have mixed performance. Specifically, our proposed supervised term weighting method, tf.rf, has a consistently better performance than other term weighting methods while other supervised term weighting methods based on information theory or statistical metric perform the worst in all experiments. On the other hand, the popularly used tf.idf method has not shown a uniformly good performance in terms of different data sets.

Dynamic Vector-Space Model for Internet Textual Information Categorization

An Improved Text Categorization Algorithm Based on VSM

DEEP WEB DATA SOURCES CLASSIFICATION BASED ON TEXT VSM OF QUERY INTERFACE

A multiclass classification framework for document categorization

Fast text categorization based on collaborative work in the semantic and class spaces

WordNet-based Concept Vector Space Model for Text Classification

An Information Retrieval Model Based On Vector Space Method By Supervised Learning

Design and analysis of a general vector space model for data classification in Internet of Things

A VECTOR SPACE MODEL BASED DOCUMENT CLASSIFICATION SYSTEM [J]

Text Representations for Text Categorization: A Case Study in Biomedical Domain

Research on Chinese Text Automatic Categorization Based on VSM

Domain-specific website recognition using hybrid vector space model

Multi View Text Categorization Based on Random Forests

A New Text Categorization Method Based on HMM and SVM

Supervised and Traditional Term Weighting Methods for Automatic Text Categorization

A Non-VSM kNN algorithm for text classification

Aggressive Dimensionality Reduction With Reinforcement Local Feature Selection For Text Categorization

Extended Information Inference Model for Unsupervised Categorization of Web Short Texts

A research on web resources automatic classification using SVMs

Using boosting mechanism to refine the threshold of VSM-based similarity in text classification

The heavy frequency vector-based text clustering