Abstract:Data stream classification for intrusion detection poses at least three major challenges. First, these data streams are typically infinite-length, making traditional multipass learning algorithms inapplicable. Second, they exhibit significant concept-drift as attackers react and adapt to defenses. Third, for data streams that do not have any fixed feature set, such as text streams, an additional feature extraction and selection task must be performed. If the number of candidate features is too large, then traditional feature extraction techniques fail. In order to address the first two challenges, this article proposes a multipartition, multichunk ensemble classifier in which a collection of v classifiers is trained from r consecutive data chunks using v-fold partitioning of the data, yielding an ensemble of such classifiers. This multipartition, multichunk ensemble technique significantly reduces classification error compared to existing single-partition, single-chunk ensemble approaches, wherein a single data chunk is used to train each classifier. To address the third challenge, a feature extraction and selection technique is proposed for data streams that do not have any fixed feature set. The technique's scalability is demonstrated through an implementation for the Hadoop MapReduce cloud computing architecture. Both theoretical and empirical evidence demonstrate its effectiveness over other state-of-the-art stream classification techniques on synthetic data, real botnet traffic, and malicious executables.

Study on an Ensemble Classification Algorithm for Data Streams with Cloud Computing

Study of Data Stream Classification Algorithm with On-demand Ensemble

Study on the Cost-Sensitive Ensemble Learning Algorithm Based on the Cloud Computing Platform

Htme: A Data Streams Processing Strategy Based On Hoeffding Tree In Mapreduce Environment

A Feature Weighted Ensemble Classifier on Stream Data

Research of Cloud Computing Based on the Hadoop Platform

A Data Streams Analysis Strategy Based on Hoeffding Tree with Concept Drift on Hadoop System

An Ensemble Classifier Framework for Mining Imbalanced Data Streams

A Data Streams Analysis Strategy Based on Hadoop Scheduling Optimization for Smart Grid Application.

A Cloud Computing Model Based on Hadoop with an Optimization of Its Task Scheduling Algorithms.

Cloud-based malware detection for evolving data streams

Research on Decision Tree Classification Algorithm of Distributed Data Stream

An Ensemble Classifier Algorithm for Mining Data Streams Based on Concept Drift

A Micro-cluster Based Ensemble Approach for Classifying Distributed Data Streams

Parallel Based on Cloud Computing to Achieve Large Data Sets Clustering

Design and implementation of parallel statiatical algorithm based on Hadoop's MapReduce model

An Ensemble Classifier Method for Classifying Data Streams with Recurrent Concept Drift.

SSC: Cloud-Based Data Stream Management in Distributed Environments

Research on the key technologies for classification of distributed data streams

Study and Implementation of Elastic Stream Computing in the Cloud

A Solution For Mining Big Data Based On Distributed Data Streams And Its Classifying Algorithms