Abstract:Online monitoring user cardinalities (or degrees) in graph streams is fundamental for many applications. For example in a bipartite graph representing user-website visiting activities, user cardinalities (the number of distinct visited websites) are monitored to report network anomalies. These real-world graph streams may contain user-item duplicates and have a huge number of distinct user-item pairs, therefore, it is infeasible to exactly compute user cardinalities when memory and computation resources are <a class="link-external link-http" href="http://limited.Existing" rel="external noopener nofollow">this http URL</a> methods are designed to approximately estimate user cardinalities, whose accuracy highly depends on parameters that are not easy to set. Moreover, these methods cannot provide anytime-available estimation, as the user cardinalities are computed at the end of the data stream. Real-time applications such as anomaly detection require that user cardinalities are estimated on the fly. To address these problems, we develop novel bit and register sharing algorithms, which use a bit array and a register array to build a compact sketch of all users' connected items respectively. Compared with previous bit and register sharing methods, our algorithms exploit the dynamic properties of the bit and register arrays (e.g., the fraction of zero bits in the bit array at each time) to significantly improve the estimation accuracy, and have low time complexity (O(1)) to update the estimations each time they observe a new user-item pair. In addition, our algorithms are simple and easy to use, without requirements to tune any parameter. We evaluate the performance of our methods on real-world datasets. The experimental results demonstrate that our methods are several times more accurate and faster than state-of-the-art methods using the same amount of memory.

HoppingTimer: A Near-optimal Framework for Basic Estimation of Data Streams in Hopping Windows

A New Algorithm for Frequency Tendency Prediction over Data Streams

Continuous Monitoring of Distributed Data Streams over a Time-Based Sliding Window

HoppingSketch: More Accurate Temporal Membership Query and Frequency Query

Novel structures for counting frequent items in time decayed streams

Efficient Discovery of Emerging Frequent Patterns in ArbitraryWindows on Data Streams

Flattened Exponential Histogram for Sliding Window Queries over Data Streams

Fine-grained Probability Counting for Cardinality Estimation of Data Streams.

An Adaptive Window Time-Frequency Analysis Method Based on Short-Time Fourier Transform

Estimation and maintenance of frequent pattern on data streams

A Sketch Framework for Approximate Data Stream Processing in Sliding Windows

Error-adaptive and time-aware maintenance of frequency counts over data streams

SHE: A Generic Framework for Data Stream Mining over Sliding Windows.

Finding Frequent Items in Time Decayed Data Streams.

Attribute outlier detection over data streams

A Unified Framework for Mining Batch and Periodic Batch in Data Streams

Utilizing Dynamic Properties of Sharing Bits and Registers to Estimate User Cardinalities over Time

Out of Many We Are One

Accurately Estimating User Cardinalities and Detecting Super Spreaders over Time

Time-Out Bloom Filter: a New Sampling Method for Recording More Flows

Detecting Change in Data Stream: Using Sampling Technique