Abstract:The Web has become a large platform for information publishing and consuming. Web news and blog are both representative information sources providing convenient ways to keep informed. In addition to the main content, most web pages also contain navigation panels, advertisements, recommended articles etc. Effectively extracting news and blog content and filtering these noises is necessary and challenging. In this paper we propose a news and blog content extraction approach that is portable to different languages and various domains. Our extensive case studies shows that characters which are not anchor texts but contain stop words are more likely to be the genuine content. Our method first traverses the entire DOM tree and count these valid characters attached to each DOM node. Then we step into the most representative child node based on valid characters recursively. And we finally stop at the main content node with a predefined criterion. To validate the approach, we conduct experiments by using online news and blog files randomly selected from well-known Chinese and English websites. Experimental result shows that our method achieves 96% F-1-measure on average and outperforms CETR.

Design and Implementation of a Web News Extraction System.

Design and Implementation of Web News Automatically Gathering and Publishing System

A generic Web news extraction approach

News Information Extraction for Web Resource

An efficient method for extracting web news content

Information extraction for eb resource

Learning to Extract Web News Title in Template Independent Way

The Design and Implementation of Configurable News Collection System Based on Web Crawler

Extracting Content For News Web Pages Based On Dom

A Novel Approach To Automatically Extracting Main Content of Web News

STATISTICS-BASED AUTOMATIC WEB NEWS TEXT EXTRACTION

Design and Application of a News Gathering and Analysis System Based on Text Clustering

Design and implementation of news collecting and filtering system based on RSS.

The research and implementation of web information extraction technology based on multi-level pages

NET Based Web Information Extraction Method

A Template Independent Approach for Web News and Blog Content Extraction

Hybrid method for automated news content extraction from the web

The Design and Implementation of a System for Retrieving Massive Information and Sharing Services

Automatic Web News Extraction Using Blocking Tag

Research on Automatic Extraction Technology of Web Information

THE DESIGN AND IMPLEMENTATION OF A GRID-ENABLED WEB INFORMATION EXTRACTION SYSTEM