Research on the Recognizing Algorithm of Web Pages

韩彬斌,王培康

DOI: https://doi.org/10.3969/j.issn.1000-0135.2001.01.012

2001-01-01

Abstract:Text information mining on WWW is a new trend in networkinformation searching field. We apply two machine learning algorithms——Rocchio algorithm and Widrow-Hoff algorithm to recognize Web pages and compare them to some custom algorithms.

What problem does this paper attempt to address?

Web Page Topic Recognition Algorithm based on Ensemble Learning

Ge Dongmou,Zhang Gang,Li Qian

DOI: https://doi.org/10.3969/j.issn.1000-8519.2013.19.023

2013-01-01

Abstract:Topic automatic recognition in large amount of web pages is an important research subfield in web-based information analysis and mining.It has important both theoretical and applicable sense.This paper proposed a web page topic recognition algorithm framework based on ensemble learning.It constructed indi- vidual base learners through heterogeneous properties set in a largest separation margin meaning,and ap- plied ensemble learning to combine the results from individual learners.The proposed algorithm is evaluated on a benchmark data and the results illustrates its effectiveness.
Research on Web Mining Technique Facing Electronic Business and Application

DONG De-min,HE Qin-ming

DOI: https://doi.org/10.3969/j.issn.1000-7024.2006.01.029

2006-01-01

Abstract:Enterprise and organize face a series of problems to capture information of web.Technique of solving these problems were summed up including web content mining,web structure mining and web usage mining.And some application of web mining was intro-duced,such as intelligent search engine,enterprise crisis management,client relation management and personalization service system.
The Research of Web Mining

孙茂松,沈达阳

DOI: https://doi.org/10.3969/j.issn.1002-137X.2000.02.021

2000-01-01

Computer Science

Abstract:The application of knowledge mining technology,is the key to improve the information searching ability of World Wide Web. Atming at different types of information,the paper introduces several Web mining technologies,which are applied in ParaSite,DRS and WebKB respectively,after comparing their characteristics,an idea of building common Web mining systsem is presented.
Ontology-driven Recognition of Topic-oriented Web Pages Relation

张合,王晓东,杨照岩

DOI: https://doi.org/10.3969/j.issn.1000-2367.2009.02.017

2009-01-01

Abstract:This paper applies ontology in web content mining,and proposes the algorithm that automatically generates and optimizes regulations which discern the relation of web pages based on ontology,recognizing the relation of web pages automatically based on knowing the class of them.This has laid the foundation for semanticizing web resources.
Mining Topic-specific Knowledge on Web

Xiong Chao-song,Lu Gan

2006-01-01

Abstract:In order to help the user on Web to search and edit topic-specific knowledge,this paper gives one kind of mining algorithm based on the sub-topic concept .The basic thought is: Assigning a topic, discovering the sub-topic or core concept in the page set returned by search engine,obtaining pages of containing the topic-specific and sub-topic concept,causing the user not to need to browse over all pages to be able to gain the topic-specific,systematic, comprehensive knowledge.
Research and Evaluation of Near-replicas of Web Pages Detection Algorithms

Jian-yong WANG,Zheng-mao XIE,Ming LEI,Xiao-ming LI

DOI: https://doi.org/10.3321/j.issn:0372-2112.2000.Z1.033

2000-01-01

Tien Tzu Hsueh Pao/Acta Electronica Sinica

Abstract:Many documents are replicated across the World-wide Web. How to efficiently and accurately find the near-replicas of web pages becomes an important topic in the search engine research area, which can be used to improve the quality of searching service. We propose five near-replicas detection algorithms for search engines that rely on keyword matching, and evaluate them using the WebGather search engine system. In addition, we also compare our method with one of the most popular copy detection mechanisms. Our method is successfully adopted to remove the near-replicas of web pages in WebGather, and it can also be widely used to build library.
AN EFFICIENT AND MULTI-PURPOSE ALGORITHM FOR MINING WEB LOGS

宋擒豹,沈钧毅

2001-01-01

Journal of Computer Research and Development

Abstract:Similar customer groups, relevant Web pages, and frequent accesspaths can be discovered by analyzing of Web log files and customer database. In this paper, novel Web log mining algorithms are presented. First, according to Web site's directed graph defined, a URL-UserID relevant matrix is set up, where URL is taken as row and UserID is taken as column, and each element's value of this matrix is the user's hits. Second, similar customer groups are discovered by measuring similarity between column vectors, and relevant Web pages are obtained by measuring similarity between row vectors; frequent access paths can also be discovered by further processing of the latter. Experiments show the effectiveness of the algorithms.
Research on Retrieving Main Segmentation of Web Pages in E-Travel

BAI He,ZHAO Zhi-qiang,WANG Jin-lin

DOI: https://doi.org/10.3969/j.issn.2095-6835.2010.15.005

2010-01-01

Abstract:Web information retrieval is key technology of E-Travel.The Main Semantic Segmentation includes the main information of a web page,the precision of its retrieval is the basis of the web page analysis。 Based on the conclusion of web page segmentation algorithms,an approach to retrieving main semantic block with machine learning method was proposed,after that,a verify on positive unit was applied to locate the best main node.This approach has achieved a good effect.
Research on Web Pages Automatic Classification Based on Internet News Corpus

CAI Wei,WANG Ying-lin,YIN Zhong-hang

IF: 8.1

2007-01-01

Information Sciences

Abstract:Web pages contain more abundant contents than pure text ,such as hyperlinks,html tags and metadata et al.So that Web page categorization is different from pure text. According to Internet Chinese news pages, a practical algorithm for extracting subject concepts from web page without thesaurus was proposed, when incorporated these category-subject concepts into knowledge base, Web pages was classified by hybrid algorithm, with experiment corpus extracting from Xinhua net. Experimental result shows that the categorization performance is improved using Web page feature.
Web Pages Clustering and Concepts Mining: an Approach Towards Intelligent Information Retrieval

Fang Li,Martin Mehlitz,Li Fen,Huanye Sheng

DOI: https://doi.org/10.1109/iccis.2006.252315

2006-01-01

Abstract:The amount of information on the Web is growing at an exponential rate. Information overload has brought a heavy burden for modern life. Keyword based search engines no long fill the needs of many people. This paper introduces an approach towards intelligent information retrieval by providing clustered Web pages and minded concepts based on results of search engines. Web page clustering is based on SVD (singular value decomposition), concepts mining is implemented with a revision of Apriori algorithm. Experiments on three different kinds of keyword as queries to information retrieval have showed a promising result
Recognition of Web Page Relation Based on Ontology

FAN Wen-Bin,DING Yan,PAN Jin-Gui

DOI: https://doi.org/10.3969/j.issn.1002-137X.2005.11.011

2005-01-01

Computer Science

Abstract:This paper analyzes semantic relation among large amount of web pages in WWW, which can realize intelli- gent query and provide characteristic services when applied to search engine When recognizing the relation of web pa- ges, this paper realizes the automatic production of rules through relation entity and sample sets defined in Ontology and recognition based on rules. Based on the idea of FOIL and the method of SRV, we reduce the training process of rules when training rules through sample sets. Through regulating and comparing the valve of characteristic selection, we can get the optimization rule sets and simplify the process of judgment when precision is improved.
Recognition of the Topic-oriented Web Page Relations Based on Ontology

樊闻斌,王守芳,金浩,潘金贵

DOI: https://doi.org/10.3321/j.issn:1000-565X.2004.z1.008

2004-01-01

Abstract:The analysis of the semantic relations among large amount of Web pages with the same topic in WWW can realize the intelligent query and provide characteristic services in search engines. According to the relation entities in Ontology, the automatic recognition of the relations among Web pages is implemented based on the Web page classification. Also, the automatic generation and optimization method for the relation recognition rules is proposed. It is found that the application of the recognition process to Dolphin system ensures users to rapidly find the Web pages, which have a semantic relation with the initiative page, thus saving the cost of browsing Web pages and finding a suitable one.
Research on the Design and Implementation of Web Mining System

乔智勇,刘志镜

DOI: https://doi.org/10.3969/j.issn.1000-7024.2002.07.012

2002-01-01

Abstract:Information overloading is a big problem in the global informatization. The web is huge, but it is difficult to find what we needed on it. We interactive with the web by click or search engines, however neither can helps us get what we want ac-curately and immediately. The web mining techniques can resolve these problems. The theories of web mining are discussed in this paper. Based on the objects, we classify web mining into three categories, namely web content mining, web structure mining, and web usage mining. Finally, we proposed a general frame of web mining systems based on the specific structures of HTML pages. The implementation is also discussed in details.
Web Mining Research: A Survey

Raymond Kosala,Hendrik Blockeel

DOI: https://doi.org/10.48550/arXiv.cs/0011033

IF: 5.414

2000-11-22

Machine Learning

Abstract:With the huge amount of information available online, the World Wide Web is a fertile area for data mining research. The Web mining research is at the cross road of research from several research communities, such as database, information retrieval, and within AI, especially the sub-areas of machine learning and natural language processing. However, there is a lot of confusions when comparing research efforts from different point of views. In this paper, we survey the research in the area of Web mining, point out some confusions regarded the usage of the term Web mining and suggest three Web mining categories. Then we situate some of the research with respect to these three categories. We also explore the connection between the Web mining categories and the related agent paradigm. For the survey, we focus on representation issues, on the process, on the learning algorithm, and on the application of the recent works as the criteria. We conclude the paper with some research issues.
Automatic Web Information Extraction Based On Rules

Fanghuai Hu,Tong Ruan,Zhiqing Shao,Jun Ding

DOI: https://doi.org/10.1007/978-3-642-24434-6_21

2011-01-01

Abstract:Web Information Extraction is the initial step of effective web mining. In this article a few heuristic rules which describe the characteristics of the main content of web pages are summarized. The rules are constructed by some pre-defined terms and metrics, which can be considered as reusable and extensible for different kinds of HTML pages. Afterwards, a probabilistic model which utilizes the rules and metrics is suggested and the corresponding algorithm is implemented. The algorithm is tested on 1000 randomly selected web pages. The experiment shows that the algorithm is more precise and more applicable to the diverse structure of different web sites than other algorithms.
Web-mining Algorithm for Recommendation System:Air

ZHANG Tao,DING Er-yu,LUO Bin

DOI: https://doi.org/10.3778/j.issn.1002-8331.2008.01.053

2008-01-01

Abstract:Internet users often spend much time finding useful pages.Recommendation system does such a job that it can help user locate information and increase the users’ loyalty.In respect that web mining is good at dealing with massive data and sparse data,web mining is widely used in recommendation system.Most important technologies in web mining include clustering,association rules,sequence pattern.Howerver,these technologies have a limitation that it is difficult to strike the balance of precision and coverage.In this paper,we integrate these technologies and raise a new algorithm(AIR).Our experiment data,performed on real usage data,indicate that AIR can achieve dramatic improvements in terms of recommendation effectiveness.
Automatic Recognition of News Web Pages

Zhu Zhu,Gong-Qing Wu,Xindong Wu,Xue-Gang Hu,Fei-Yue Wang

DOI: https://doi.org/10.1007/978-3-540-69304-8_52

2008-01-01

Abstract:The information on the World Wide Web is congested with large amounts of news contents. The filtering, summarization and classification of news Web pages have become hot topics of research, aiming for useful news contents. Accurately identifying news Web pages is a crucial problem in these research topics. To solve this problem, this paper proposes an automatic recognition method for news Web pages based on a combination of URL attributes, structure attributes and content attributes. Our experimental results demonstrate that this method provides a high accuracy of above 96% with the recognition of news Web page.
Web Document Classification Techniques

孙建涛,沈抖,陆玉昌,石纯一

DOI: https://doi.org/10.3321/j.issn:1000-0054.2004.01.017

2004-01-01

Abstract:Web document classification assigns labels to web documents based on machine learning techniques. A review of various text classification techniques showed that the main difficulties in web document classification are the page representation methods and the classification algorithms. Techniques that go beyond text categorization approaches are needed. Probabilistic algorithms and relational learning methods are both time-consuming. SVM (support vector machine) classifiers are quite accurate but the automatic kernel selection and the large scale training are both key problems. Various measures were investigated to compare algorithm performance based on sample datasets.
An Algorithm on Web Article Automatic Extraction Based on DOM Structure

Weiguo Shen,Xiaojian Zou

DOI: https://doi.org/10.14257/ijhit.2015.8.3.22

2015-01-01

International Journal of Hybrid Information Technology

Abstract:By analyzing a large number of web pages, we proposed a page segment algorithm which is based on DOM tree structural features and visual features. The algorithm segments the pages to the small particles. It produces basic processing units for recognition algorithms. After segmenting the pages, we extracted the structural and visual features of the pages, and proposed a method to identify the body of the web article. The method uses clustering algorithm and heuristic rules to produce an automatic wrapper. A testing experiment demonstrated the efficacy of the algorithm.
Algorithm for Webpage Semantic Blocks Mining Using Tree Match Method

LIU Shou-qun,ZHU Ming,TAN Xiao-bin

2009-01-01

Abstract:In the WWW,many web documents are combined with various semantic regions.Discovery and mining such regions has a significant effort for web page analysis,user browser experience improvement,etc.But because of the difference of web page structure and content among large amounts of web pages,it is hard to detect such common regions effectively and correctly,traditional matching methods such as regular expression are not suitable for this problem.This paper proposes a region detection method based on tree match algorithm.As is shown according the experiments,the method this paper described improves F-Measure value,besides this method also reduces computation cost.

Research on the Recognizing Algorithm of Web Pages

Web Page Topic Recognition Algorithm based on Ensemble Learning

Research on Web Mining Technique Facing Electronic Business and Application

The Research of Web Mining

Ontology-driven Recognition of Topic-oriented Web Pages Relation

Mining Topic-specific Knowledge on Web

Research and Evaluation of Near-replicas of Web Pages Detection Algorithms

AN EFFICIENT AND MULTI-PURPOSE ALGORITHM FOR MINING WEB LOGS

Research on Retrieving Main Segmentation of Web Pages in E-Travel

Research on Web Pages Automatic Classification Based on Internet News Corpus

Web Pages Clustering and Concepts Mining: an Approach Towards Intelligent Information Retrieval

Recognition of Web Page Relation Based on Ontology

Recognition of the Topic-oriented Web Page Relations Based on Ontology

Research on the Design and Implementation of Web Mining System

Web Mining Research: A Survey

Automatic Web Information Extraction Based On Rules

Web-mining Algorithm for Recommendation System:Air

Automatic Recognition of News Web Pages

Web Document Classification Techniques

An Algorithm on Web Article Automatic Extraction Based on DOM Structure

Algorithm for Webpage Semantic Blocks Mining Using Tree Match Method