Abstract:To help the growing qualitative and quantitative demands for information from the WWW, efficient automatic Web page classifiers are urgently needed. However, a classifier applied to the WWW faces a huge-scale dimensionality problem since it must handle millions of Web pages, tens of thousands of features, and hundreds of categories. When it comes to practical implementation, reducing the dimensionality is a critically important challenge. In this paper, we propose a fuzzy ranking analysis paradigm together with a novel relevance measure, discriminating power measure (DPM), to effectively reduce the input dimensionality from tens of thousands to a few hundred with zero rejection rate and small decrease in accuracy. The two-level promotion method based on fuzzy ranking analysis is proposed to improve the behavior of each relevance measure and combine those measures to produce a better evaluation of features. Additionally, the DPM measure has low computation cost and emphasizes on both positive and negative discriminating features. Also, it emphasizes classification in parallel order, rather than classification in serial order. In our experimental results, the fuzzy ranking analysis is useful for validating the uncertain behavior of each relevance measure. Moreover, the DPM reduces input dimensionality from 10,427 to 200 with zero rejection rate and with less than 5% decline (from 84.5% to 80.4%) in the test accuracy. Furthermore, to consider the impacts on classification accuracy for the proposed DPM, the experimental results of China Time and Reuter-21578 datasets have demonstrated that the DPM provides major benefit to promote document classification accuracy rate. The results also show that the DPM indeed can reduce both redundancy and noise features to set up a better classifier.

Using Gini-Index for Feature Selection in Text Categorization

CLDA: Feature Selection for Text Categorization Based on Constrained LDA

Feature Selection Method on Imbalanced Text

A New Approach of Feature Selection for Text Categorization

An Effective Feature Selection Method For Text Categorization

Aggressive Dimensionality Reduction With Reinforcement Local Feature Selection For Text Categorization

A Novel Feature Selection Approach and Feature Weight Adjustment Technique in Text Classification

Improving Short Text Classification Through Better Feature Space Selection

Select Strong Information Features to Improve Text Categorization Effectiveness

Learning Effective Features for Chinese Text Categorization

Relative Term-Frequency Based Feature Selection for Text Categorization

Feature Selection for Support Vector Machines in Text Categorization

A comprehensive unsupervised feature selection method of two-stage strategy

A General Framework of Feature Selection for Text Categorization

OCFS: optimal orthogonal centroid feature selection for text categorization.

A Comparative Study on Feature Weight in Text Categorization

An Evaluation on Feature Selection for Text Clustering

Text Categorization Based On. Concept Indexing and Principal Component Analysis

Toward Optimal Feature Selection in Naive Bayes for Text Categorization

Two novel feature selection approaches for web page classification

A High Performance Two-Class Chinese Text Categorization Method