Randomized Admission Policy for Efficient Top-k and Frequency Estimation

Ran Ben Basat,Gil Einziger,Roy Friedman,Yaron Kassner
DOI: https://doi.org/10.48550/arXiv.1612.02962
2016-12-09
Abstract:Network management protocols often require timely and meaningful insight about per flow network traffic. This paper introduces Randomized Admission Policy (RAP) - a novel algorithm for the frequency and top-k estimation problems, which are fundamental in network monitoring. We demonstrate space reductions compared to the alternatives by a factor of up to 32 on real packet traces and up to 128 on heavy-tailed workloads. For top-k identification, RAP exhibits memory savings by a factor of between 4 and 64 depending on the skew of the workload. These empirical results are backed by formal analysis, indicating the asymptotic space improvement of our probabilistic admission approach. Additionally, we present d-Way RAP, a hardware friendly variant of RAP that empirically maintains its space and accuracy benefits.
Data Structures and Algorithms,Networking and Internet Architecture
What problem does this paper attempt to address?