Abstract:Online monitoring user cardinalities (or degrees) in graph streams is fundamental for many applications. For example in a bipartite graph representing user-website visiting activities, user cardinalities (the number of distinct visited websites) are monitored to report network anomalies. These real-world graph streams may contain user-item duplicates and have a huge number of distinct user-item pairs, therefore, it is infeasible to exactly compute user cardinalities when memory and computation resources are <a class="link-external link-http" href="http://limited.Existing" rel="external noopener nofollow">this http URL</a> methods are designed to approximately estimate user cardinalities, whose accuracy highly depends on parameters that are not easy to set. Moreover, these methods cannot provide anytime-available estimation, as the user cardinalities are computed at the end of the data stream. Real-time applications such as anomaly detection require that user cardinalities are estimated on the fly. To address these problems, we develop novel bit and register sharing algorithms, which use a bit array and a register array to build a compact sketch of all users' connected items respectively. Compared with previous bit and register sharing methods, our algorithms exploit the dynamic properties of the bit and register arrays (e.g., the fraction of zero bits in the bit array at each time) to significantly improve the estimation accuracy, and have low time complexity (O(1)) to update the estimations each time they observe a new user-item pair. In addition, our algorithms are simple and easy to use, without requirements to tune any parameter. We evaluate the performance of our methods on real-world datasets. The experimental results demonstrate that our methods are several times more accurate and faster than state-of-the-art methods using the same amount of memory.

SRLA: A Real Time Sliding Time Window Super Point Cardinality Estimation Algorithm for High Speed Network Based on GPU

Memory efficient distributed sliding super point cardinality estimation by GPU

VATE: a trade-off between memory and preserving time for high accuracy cardinalities estimation under sliding time window

Cardinalities estimation under sliding time window by sharing HyperLogLog Counter

Achieving Real-Time Quality of Service with Software Defined Networking

Most memory efficient distributed super points detection on core networks

GAM: A GPU-Accelerated Algorithm for MaxRS Queries in Road Networks

READ: a three-communicating-stage distributed super points detections algorithm

A novel spread estimation based abnormal flow detection in high-speed networks

Accurately Estimating User Cardinalities and Detecting Super Spreaders over Time

A holistic approach to build real-time stream processing system with GPU

Network-Load Estimation for K-Repetition Grant-Free Access Enabling Adaptive Resource Allocation Towards QoS Enhancement

A GPU-Based Solution to Fast Calculation of Betweenness Centrality on Large Weighted Networks

FastSO: A Fast Weighted Cardinality Estimation Algorithm

A Data Streaming Method for Monitoring Host Connection Degrees of High-Speed Links

SSL : A S urrogate-Based Method for Large-Scale S tatistical Latency Measurement

Utilizing Dynamic Properties of Sharing Bits and Registers to Estimate User Cardinalities over Time

SSL: ASurrogate-Based Method for Large-ScaleStatisticalLatencyMeasurement

A time window based analytical approach for splitting high-speed network traffic

GPUSCAN: GPU-Based Parallel Structural Clustering Algorithm for Networks

Identifying Heavy Hitters in High-Speed Network Monitoring