Abstract:To help the growing qualitative and quantitative demands for information from the WWW, efficient automatic Web page classifiers are urgently needed. However, a classifier applied to the WWW faces a huge-scale dimensionality problem since it must handle millions of Web pages, tens of thousands of features, and hundreds of categories. When it comes to practical implementation, reducing the dimensionality is a critically important challenge. In this paper, we propose a fuzzy ranking analysis paradigm together with a novel relevance measure, discriminating power measure (DPM), to effectively reduce the input dimensionality from tens of thousands to a few hundred with zero rejection rate and small decrease in accuracy. The two-level promotion method based on fuzzy ranking analysis is proposed to improve the behavior of each relevance measure and combine those measures to produce a better evaluation of features. Additionally, the DPM measure has low computation cost and emphasizes on both positive and negative discriminating features. Also, it emphasizes classification in parallel order, rather than classification in serial order. In our experimental results, the fuzzy ranking analysis is useful for validating the uncertain behavior of each relevance measure. Moreover, the DPM reduces input dimensionality from 10,427 to 200 with zero rejection rate and with less than 5% decline (from 84.5% to 80.4%) in the test accuracy. Furthermore, to consider the impacts on classification accuracy for the proposed DPM, the experimental results of China Time and Reuter-21578 datasets have demonstrated that the DPM provides major benefit to promote document classification accuracy rate. The results also show that the DPM indeed can reduce both redundancy and noise features to set up a better classifier.

Method of acquiring web features and its application in web search

Exploiting Multi-Category Characteristics and Unified Framework to Extract Web Content

Extracting Novel Features for E-Commerce Page Quality Classification.

Extracting Novel Features for E-Commerce Page Quality Classification

A Novel Method for Bilingual Web Page Acquisition from Search Engine Web Records.

Information Retrieval from Large Number of Web Sites

Research on Intelligent Information Search Based on Web

Keyword Extraction Based on Multi-feature Fusion for Chinese Web Pages

Still Haven't Found What You're Looking For -- Detecting the Intent of Web Search Missions from User Interaction Features

Research on Feature Extraction for Content-based Chinese Web Pages Analysis

A Weakly Supervised Academic Search Model Based on Knowledge-Enhanced Feature Representation

Web Page Content Extraction Based on Multi-feature Fusion

Web Pages Information Retrieval Based on Keywords Cluster and Node Instance

A Novel Method for Bilingual Web Page Mining Via Search Engines

Acquiring User Information Needs for Personalized Search

Extraction of Relevant Snippets from Web Pages Using Hybrid Features.

Learning Deep Web Crawling with Diverse Features

Improve the performance of web pages relevance judgment using query-biased summary

Two novel feature selection approaches for web page classification

Webpage retrieval based on query by example for think tank construction

A New Keywords Method to Improve Web Search