Abstract:Real-time surveillance systems, telecommunication systems, and other dynamic environments often generate tremendous (potentially infinite) volume of stream data: the volume is too huge to be scanned multiple times. Much of such data resides at rather low level of abstraction, whereas most analysts are interested in relatively high-level dynamic changes (such as trends and outliers). To discover such high-level characteristics, one may need to perform on-line multi-level, multi-dimensional analytical processing of stream data. In this paper, we propose an architecture, called stream_cube, to facilitate on-line, multi-dimensional, multi-level analysis of stream data. For fast online multi-dimensional analysis of stream data, three important techniques are proposed for efficient and effective computation of stream cubes. First, a tilted time frame model is proposed as a multi-resolution model to register time-related data: the more recent data are registered at finer resolution, whereas the more distant data are registered at coarser resolution. This design reduces the overall storage of time-related data and adapts nicely to the data analysis tasks commonly encountered in practice. Second, instead of materializing cuboids at all levels, we propose to maintain a small number of critical layers. Flexible analysis can be efficiently performed based on the concept of observation layer and minimal interesting layer . Third, an efficient stream data cubing algorithm is developed which computes only the layers (cuboids) along a popular path and leaves the other cuboids for query-driven, on-line computation. Based on this design methodology, stream data cube can be constructed and maintained incrementally with a reasonable amount of memory, computation cost, and query response time. This is verified by our substantial performance study.

Maintaining Aggregate Views over Streams

Mining Fuzzy Association Rules in Data Streams

An Efficient Approach of Processing Multiple Continuous Queries

Progressive online aggregation in a distributed stream system

Approximate continuous querying over distributed streams

Quality-driven processing of sliding window aggregates over out-of-order data streams

A Cube Analytical Mining Framework for Stream Data

Sliding-Window Probabilistic Threshold Aggregate Queries on Uncertain Data Streams

Effective Skyline Cardinality Estimation on Data Streams

An Evaluation of Streaming Algorithms for Distinct Counting Over a Sliding Window

Stream Cube: an Architecture for Multi-Dimensional Analysis of Data Streams

Out-of-Order Sliding-Window Aggregation with Efficient Bulk Evictions and Insertions (Extended Version)

Load Shedding for Shared Window Joins over Data Streams

Optimizing Cost of Continuous Overlapping Queries over Data Streams by Filter Adaption

Design of a Sliding Window over Asynchronous Event Streams

Approximate Query on Historical Stream Data

Sliding Window Top-K Monitoring over Distributed Data Streams

Continuous Monitoring of Distributed Data Streams over a Time-Based Sliding Window

Design of a Sliding Window over Distributed and Asynchronous Event Streams

Sliding Sketches: A Framework Using Time Zones for Data Stream Processing in Sliding Windows

Parallelizing skyline queries over uncertain data streams with sliding window partitioning and grid index