Abstract:Real-time surveillance systems, telecommunication systems, and other dynamic environments often generate tremendous (potentially infinite) volume of stream data: the volume is too huge to be scanned multiple times. Much of such data resides at rather low level of abstraction, whereas most analysts are interested in relatively high-level dynamic changes (such as trends and outliers). To discover such high-level characteristics, one may need to perform on-line multi-level, multi-dimensional analytical processing of stream data. In this paper, we propose an architecture, called stream_cube, to facilitate on-line, multi-dimensional, multi-level analysis of stream data. For fast online multi-dimensional analysis of stream data, three important techniques are proposed for efficient and effective computation of stream cubes. First, a tilted time frame model is proposed as a multi-resolution model to register time-related data: the more recent data are registered at finer resolution, whereas the more distant data are registered at coarser resolution. This design reduces the overall storage of time-related data and adapts nicely to the data analysis tasks commonly encountered in practice. Second, instead of materializing cuboids at all levels, we propose to maintain a small number of critical layers. Flexible analysis can be efficiently performed based on the concept of observation layer and minimal interesting layer . Third, an efficient stream data cubing algorithm is developed which computes only the layers (cuboids) along a popular path and leaves the other cuboids for query-driven, on-line computation. Based on this design methodology, stream data cube can be constructed and maintained incrementally with a reasonable amount of memory, computation cost, and query response time. This is verified by our substantial performance study.

Cube-based Incremental Outlier Detection for Streaming Computing

A Cluster-Based Incremental Recommendation Algorithm on Stream Processing Architecture

Continuous Angle-based Outlier Detection on High-dimensional Data Streams.

Real-time distance-based outlier detection in data streams

Stream Cube: an Architecture for Multi-Dimensional Analysis of Data Streams

CELOF: Effective and fast memory efficient local outlier detection in high-dimensional data streams

Improved incremental local outlier detection for data streams based on the landmark window model

A Review of Local Outlier Factor Algorithms for Outlier Detection in Big Data Streams

FCI-Outlier: An Efficient Frequent Closed Itemset-Based Outlier Detecting Approach on Data Stream

Robust High-dimensional Bioinformatics Data Streams Mining by ODR-ioVFDT

Attribute outlier detection over data streams

A Cube Analytical Mining Framework for Stream Data

Distributed Outlier Detection Using Compressive Sensing

ELOF: fast and memory-efficient anomaly detection algorithm in data streams

Continuous Outlier Mining of Streaming Data in Flink

A scalable supervised algorithm for dimensionality reduction on streaming data

Incremental Subspace Clustering over Multiple Data Streams

Outlier Detection in Streaming Data for Telecommunications and Industrial Applications: A Survey

An efficient FPGA implementation of Mahalanobis distance-based outlier detection for streaming data

On the Improvement of the Isolation Forest Algorithm for Outlier Detection with Streaming Data

Dynamic Micro-cluster-Based Streaming Data Clustering Method for Anomaly Detection.