Abstract:In recent years, detecting anomalies in real-world computer networks has become a more and more challenging task due to the steady increase of high-volume, high-speed and high-dimensional streaming data, for which ground truth information is not available. Efficient detection schemes applied on networked embedded devices need to be fast and memory-constrained, and must be capable of dealing with concept drifts when they occur. Different approaches for unsupervised online outlier detection have been designed to deal with these circumstances in order to reliably detect malicious activity. In this paper, we introduce a novel framework called PCB-iForest, which generalized, is able to incorporate any ensemble-based online OD method to function on streaming data. Carefully engineered requirements are compared to the most popular state-of-the-art online methods with an in-depth focus on variants based on the widely accepted isolation forest algorithm, thereby highlighting the lack of a flexible and efficient solution which is satisfied by PCB-iForest. Therefore, we integrate two variants into PCB-iForest—an isolation forest improvement called extended isolation forest and a classic isolation forest variant equipped with the functionality to score features according to their contributions to a sample’s anomalousness. Extensive experiments were performed on 23 different multi-disciplinary and security-related real-world datasets in order to comprehensively evaluate the performance of our implementation compared with off-the-shelf methods. The discussion of results, including AUC, F1 score and averaged execution time metric, shows that PCB-iForest clearly outperformed the state-of-the-art competitors in 61% of cases and even achieved more promising results in terms of the tradeoff between classification and computational costs.

Frod: Fast And Robust Distance-Based Outlier Detection With Active-Inliers-Patterns In Data Streams

Continuous Angle-based Outlier Detection on High-dimensional Data Streams.

Real-time distance-based outlier detection in data streams

Attribute outlier detection over data streams

FCI-Outlier: An Efficient Frequent Closed Itemset-Based Outlier Detecting Approach on Data Stream

Robust High-dimensional Bioinformatics Data Streams Mining by ODR-ioVFDT

An Algorithm for Data Stream Speed Anomaly Detection Based on RCSW

Outlier Detection in Streaming Data for Telecommunications and Industrial Applications: A Survey

CELOF: Effective and fast memory efficient local outlier detection in high-dimensional data streams

An efficient approach for outlier detection from uncertain data streams based on maximal frequent patterns

A Review of Local Outlier Factor Algorithms for Outlier Detection in Big Data Streams

Cube-based Incremental Outlier Detection for Streaming Computing

Improved incremental local outlier detection for data streams based on the landmark window model

RTOD: Efficient Outlier Detection with Ray Tracing Cores

An efficient FPGA implementation of Mahalanobis distance-based outlier detection for streaming data

Continuous Outlier Mining of Streaming Data in Flink

ELOF: fast and memory-efficient anomaly detection algorithm in data streams

On the Improvement of the Isolation Forest Algorithm for Outlier Detection with Streaming Data

Dynamic Micro-cluster-Based Streaming Data Clustering Method for Anomaly Detection.

UWFP-Outlier: an efficient frequent-pattern-based outlier detection method for uncertain weighted data streams

A Fast Outlier Detection Method for Big Data.