Abstract:Focused Crawler searches the internet for topic-specific web pages. Its effectiveness is determined on the multidimensional nature of the web pages. The main task of any Focused Crawler is to collect relevant web pages of predefined topics and neglecting the irrelevant web pages. Traditional Best-First based Focused Crawlers (FC) are based on Vector Space Model (VSM) which uses Term Frequency-Inverse Document Frequency (TF-IDF) that gives limited success rate on the web page classification. The major practical challenge associated with Focused Crawler is to correctly classify the web pages based on the given topic due to the unstructured data in web pages. The main objective of this work is to design an improved focused Crawling approach using web page classification. This work proposes a text classification model based on Long Short Term Memory (LSTM) and Convolutional Neural Network (CNN) with word embeddings to increase the accuracy of web page classification. The LSTM-CNN based text classification model is further used to guide the Focused Crawler for classification of web pages. The proposed text classification model is implemented by combining the LSTM with CNN. The validation of the proposed LSTM-CNN text classification model is carried out on different datasets and results are then compared with traditional supervised machine learning algorithms and different deep neural network (DNN) based approaches like CNN, RNN and RCNN.The suggested text classification model performs 8–12 percent better than typical supervised machine learning algorithms and 4–6 percent better than CNN, RNN, and RCNN, according to experimental results. Also, the improved focused crawling approach with LSTM-CNN based text classification model gives increasing harvest rate and target recall as compared to the Breadth-First Crawler, Best-First Crawler,CNN Crawler and DNN Crawler.

A Web Information Extraction Framework with Adaptive and Failure Prediction Feature

A Domain Adaptation Framework for Short-term Traffic Prediction.

A Failure Prediction Approach Based on BiLSTM and Deep Feature Extractor for Hard Disk Drives.

Deep Neural Networks for Web Page Information Extraction

Automatic Remaining Useful Life Estimation Framework with Embedded Convolutional LSTM as the Backbone

Web Page Content Extraction Based on Multi-feature Fusion

A Machine Learning Framework for Combined Information Extraction and Integration

A Knowledge Extraction Framework for Domain-Specific Application with Simplified Pre-Trained Language Model and Attention-Based Feature Extractor

A Joint Learning Framework for Restaurant Survival Prediction and Explanation.

Machine Remaining Useful Life Prediction Via an Attention-Based Deep Learning Approach

Automated predictive modeling and framework

Adaptive deep learning-based remaining useful life prediction framework for systems with multiple failure patterns

An Adaptive and Dynamical Neural Network for Machine Remaining Useful Life Prediction

Roller: a novel approach to Web information extraction

FreeDOM: A Transferable Neural Architecture for Structured Information Extraction on Web Documents

Real-Time AI-Based Informational Decision-Making Support System Utilizing Dynamic Text Sources

A Fuzzy Approach for Pertinent Information Extraction from Web Resources

Adaptive Ordered Information Extraction with Deep Reinforcement Learning

A Complex Event Processing Framework for an Adaptive Language Learning System

Online Fabric Defects Detection Using Convolutional Neural Networks with Two Frameworks

An efficient focused crawler using LSTM-CNN based deep learning