Abstract:One fundamental challenge in data stream processing is to cope with the ubiquity of disorder of tuples within a stream caused by network latency, operator parallelization, merging of asynchronous streams, etc. High result accuracy and low result latency are two conflicting goals in out-of-order stream processing. Different applications may prefer different extent of trade-offs between the two goals. However, existing disorder handling solutions either try to meet one goal to the extreme by sacrificing the other, or try to meet both goals but have shortcomings including unguaranteed result accuracy or increased complexity in operator implementation and application logic. To meet different application requirements on the latency versus result accuracy trade-off in out-of-order stream processing, in this paper, we propose to make this trade-off user-configurable. Particularly, focusing on sliding window aggregates, we introduce AQ-K-slack, a buffer-based quality-driven disorder handling approach. AQ-K-slack leverages techniques from the fields of sampling-based approximate query processing and control theory. It can adjust the input buffer size dynamically to minimize the result latency, while respecting user-specified threshold on relative errors in produced query results. AQ-K-slack requires no a priori knowledge of disorder characteristics of data streams, and imposes no changes to the query operator implementation or the application logic. Experiments over real-world out-of-order data streams show that, compared to the state-of-art, AQ-K-slack can reduce the average buffer size, thus the average result latency, by at least 51% while respecting user-specified requirement on the accuracy of query results.

Sliding-Window Probabilistic Threshold Aggregate Queries on Uncertain Data Streams

Sliding-window Top-K Queries on Uncertain Streams

Approximation Algorithms for Aggregate Queries on Uncertain Data

Sliding windows over uncertain data streams

Outlier Detection over Sliding Windows for Probabilistic Data Streams

A Framework for High-Quality Clustering Uncertain Data Stream over Sliding Windows.

Probabilistic Threshold Range Aggregate Query Processing over Uncertain Data

Efficiently Clustering Probabilistic Data Streams.

Grid Probability Density-based clustering for uncertain data streams over sliding windows

Continuous Monitoring Of Top-K Dominating Queries Over Uncertain Data Streams

Probabilistic Threshold Join Over Distributed Uncertain Data

An Algorithm for Clustering Uncertain Data Streams over Sliding Windows

Semantic Load Shedding for Sliding Window Join-Aggregation Queries over Data Streams

Quality-driven processing of sliding window aggregates over out-of-order data streams

Approximately Processing Multi-granularity Aggregate Queries over Data Streams

Probabilistic Skyline Operator over Sliding Windows.

Space Efficient Quantile Summary for Constrained Sliding Windows on a Data Stream

Predictive Queries Algorithm Based on Probability Model over Data Streams

Approximate Processing of Massive Continuous Quantile Queries over High-Speed Data Streams

Approximate Continuous Top- K Query over Sliding Window

Efficiently Answering Probabilistic Threshold Top-k Queries on Uncertain Data