Abstract:The World Wide Web is a vast and rapidly growing repository of information. There are various kinds of objects, such as products, people, conferences, and so on, embedded in both statically and dynamically generated Web pages. Extracting the information about real-world objects is a key technique for Web mining systems. For example, the object-level search engines, such as Libra (http://libra.msra.cn) and Rexa (http://rexa.info), which help researchers find academic information like papers, conferences and researcher’s personal information, completely rely on structured Web object information. However, how to extract the object information from diverse Web pages is a challenging problem. Traditional methods are mainly template-dependent and thus not scalable to the huge number of Web pages. Furthermore, many methods are based on heuristic rules. So they are not robust enough. Recent developments in statistical machine learning make it possible to develop advanced statistical Web object extraction models. One key difference of Web object extraction from traditional information extraction from natural language text documents is that Web pages have plenty of structure information, such as two-dimensional spatial layouts and hierarchical vision tree representation. Statistical Web object extraction models can effectively leverage this information with properly designed statistical models. Another challenge of Web object extraction is that many text contents on Web pages are not regular natural language sentences. They have some structures but are lack of natural language grammars. Thus, existing natural language processing (NLP) techniques are not directly applicable. Fortunately, statistical Web object extraction models can easily merge with statistical NLP methods which have been the theme in the field of natural language processing during the last decades. Thus, the structure information on Web pages can be leveraged to help process text contents, and traditional NLP methods can be used to extract more features. Finally, the Web object extraction from diverse and large-scale Web pages provides a valuable and challenging problem for machine learning researchers. To nicely solve the problem, new learning methodology and new models (Zhu et al., 2007b) have to be developed.

Military Web Information Extraction Incorporating Resource Metadata Distribution

Exploiting Multi-Category Characteristics and Unified Framework to Extract Web Content

Extracting Web Content by Exploiting Multi-Category Characteristics

Web Content Extraction Based on Maximum Continuous Sum of Text Density.

CMNEE: A Large-Scale Document-Level Event Extraction Dataset based on Open-Source Chinese Military News

Automatic Document Metadata Extraction Based on Deep Networks.

Research on Entity Relation Extraction for Military Field.

Effective Metadata Extraction from Irregularly Structured Web Content

Web Content Extraction & Its Data Management Method

WIEAS: Helping to Discover Web Information Sources and Extract Data from Them

Automatic Extraction and Cluster Analysis of Natural Disaster Metadata Based on the Unified Metadata Framework

An efficient method for extracting web news content

Automatic Elements Extraction of Chinese Web News Using Prior Information of Content and Structure

BDBRC: A Chinese Military Entity Recognition Model Combining Context Contribution and Residual Dilatation Convolutional Networks

Distributed and Cooperative Information Retrieval on the World Wide Web

Web News Extraction Via Tag Path Feature Fusion Using DS Theory

Application and Design of Web Information Extraction System Based on Pattern Discovery

Statistical Web Object Extraction

An integrated corpus-based text mining approach used to process military technical information for facilitating EFL troopers’ linguistic comprehension: US anti-tank missile systems field manual as an example

Analysis and Comparison of Web Information Extraction Technologies

A Web Site Representation and Mining Algorithm Using the Multiscale Tree Model