Abstract:Network telemetry systems provide critical visibility into the state of network traffic. By leveraging modern programmable switch hardware, significant progress has been made to scale these systems to production network traffic workloads. Less attention has been paid towards efficiently utilizing these hardware targets’ limited resources in the face of dynamics such as the composition of the traffic workload as well as the number and types of queries running at any given point in time. However, both of these dynamics have implications on resource requirements and query accuracy. Building on our prior work DynATOS, which argues that this dynamics problem motivates reframing telemetry systems as resource schedulers, we present in this paper the design, implementation, and evaluation of DynATOS+. DynATOS+ relies on the same efficient time-division approximation and scheduling algorithm that DynATOS uses and that allows for user-defined query accuracy and latency specifications that are intended to result in tradeoffs with respect to query execution to reduce hardware resource usage. However, unlike DynATOS, DynATOS+ significantly reduces the burden on end users to express their queries by allowing them to use simple-to-state accuracy goals. For example, the method for specifying per-query accuracy goals in DynATOS+ no longer requires end users to either know the average range of query results in advance or to submit multiple trial queries to tune their accuracy goal specifications. We perform extensive simulation-based evaluations that (i) show that this new functionality of DynATOS+ works in practice, (ii) illustrate in detail the tradeoffs that result with respect to query execution and hardware resource usage for a wide range of systems parameters, and (iii) allow for an assessment of system performance under changing query workloads on top of changes in the composition of traffic workloads that has eluded previous work in this area.

DynATOS$+$: A Network Telemetry System for Dynamic Traffic and Query Workloads

Distributed Network Telemetry with Resource Efficiency and Full Accuracy

Flow Anomaly Telemetry Driven by Programmable Data Plane

Toward Scalable and Low-Cost Traffic Testing for Evaluating DDoS Defense Solutions

Sonata: Query-Driven Network Telemetry

DynamiQ: Planning for Dynamics in Network Streaming Analytics Systems

OmniMon

Direct Telemetry Access

Sensor Queries: Algebraic Optimization for Time and Energy

NQ/ATP: Architectural Support for Massive Aggregate Queries in Data Center Networks

Newton: Intent-Driven Network Traffic Monitoring

Zero-CPU Collection with Direct Telemetry Access

HyperSight: Towards Scalable, High-Coverage, and Dynamic Network Monitoring Queries

NetView: Towards on-demand network-wide telemetry in the data center

Reliable and Distributed Network Monitoring via In-band Network Telemetry

Efficient Algorithm for Traffic Engineering in Multi-domain Networks.

Semi-Oblivious Traffic Engineering: The Road Not Taken

An Efficient Adaptive Denoising Sketch for Per-flow Traffic Measurement

FlowMon-DPDK: Parsimonious Per-Flow Software Monitoring at Line Rate

DAP-Sketch: An accurate and effective network measurement sketch with Deterministic Admission Policy

Application Level High Speed Transfer Optimization Based on Historical Analysis and Real-time Tuning