Abstract:Collecting flow records is a common practice of network operators and researchers for monitoring, diagnosing and understanding a network. Traditional tools like NetFlow face great challenges when both the speed and the complexity of the network traffic increase. To keep pace up, we propose HashFlow, a tool for more efficient and accurate collection and analysis of flow records. The central idea of HashFlow is to maintain accurate records for elephant flows, but summarized records for mice flows, by applying a novel collision resolution and record promotion strategy to hash tables. The performance bound can be analyzed with a probabilistic model, and with this strategy, HashFlow achieves a better utilization of space, and also more accurate flow records, without bringing extra complexity. We have implemented HashFlow, as well as several latest flow measurement algorithms such as FlowRadar, HashPipe and ElasticSketch, in a P4 software switch. Then we use traces from different operational networks to evaluate them. In these experiments, for various types of traffic analysis applications, HashFlow consistently demonstrates a clearly better performance against its state-of-the-art competitors. For example, using a small memory of 1 MB, HashFlow can accurately record around 55K flows, which is often 12.5% higher than the others. For estimating the sizes of 50K flows, HashFlow achieves a relative error of around 11.6%, while the estimation error of the best competitor is 42.9% higher. It detects 96.1% of the heavy hitters out of 250K flows with a size estimation error of 5.6%, which is 11.3% and 73.7% better than the best competitor respectively. At last, we show these merits of HashFlow come with almost no degradation of throughput.

FlowInfra: A fault-resilient scalable infrastructure for network-wide flow level measurement

An Infrastructure for Flow-Level Network-Wide Traffic Measurement Based on Netflow

RFT: Toward Highly Reliable Flow Data Transmission in Network Measurement

AdaFlow: Adaptive Control to Improve Availability of OpenFlow Forwarding for Burst Quantity of Flows.

Flow Anomaly Telemetry Driven by Programmable Data Plane

MINT: Empowering Multiple Flow Definition Query for Network-Wide Measurement.

Efficient and Accurate Flow Record Collection With HashFlow

Continuous Flow Measurement with SuperFlow.

HashFlow For Better Flow Record Collection

TIFAflow: Enhancing Traffic Archiving System with Flow Granularity for Forensic Analysis in Network Security

Exploring Flow Characteristics in IPv6: A Comparative Measurement Study with IPv4 for Traffic Monitoring.

SuperFlow: A Reliable and Scalable Architecture for Large-Scale Enterprise Networks

Flow-Level Traffic Scheduling in Data Center Networks

Floware: Balanced Flow Monitoring in Software Defined Networks

A NetFlow based flow analysis and monitoring system in enterprise networks

Superflow: A Reliable, Controllable And Scalable Architecture For Large-Scale Enterprise Networks

Fastlane-ing More Flows with Less Bandwidth for Software-Defined Networking.

FlowStar: Fast Convergence Per-Flow State Accurate Congestion Control for InfiniBand

A Flow Exporting System Building On Network Processor

FASTeller: A Hardware Partial Aggregator for Accurate Flow Counting in Cloud Networks

Rethinking Fine-grained Measurement from Software-defined Perspective: A Survey