Abstract:Benchmarking is commonly used in research fields, such as computer architecture design and machine learning, as a powerful paradigm for rigorously assessing, comparing, and developing novel technologies. However, the data centre networking community lacks a standard open-access benchmark. This is curtailing the community's understanding of existing systems and hindering the ability with which novel technologies can be developed, compared, and tested. We present TrafPy; an open-access framework for generating both realistic and custom data centre network traffic traces. TrafPy is compatible with any simulation, emulation, or experimentation environment, and can be used for standardised benchmarking and for investigating the properties and limitations of network systems such as schedulers, switches, routers, and resource managers. To demonstrate the efficacy of TrafPy, we use it to conduct a thorough investigation into the sensitivity of 4 canonical scheduling algorithms (shortest remaining processing time, fair share, first fit, and random) to varying traffic trace characteristics. We show how the fundamental scheduler performance insights revealed by these tests translate to 4 realistic data centre network types; University, Private Enterprise, Commercial Cloud, and Social Media Cloud. We then draw conclusions as to which types of scheduling policies are most suited to which types of network load conditions and traffic characteristics, leading to the possibility of application-informed decision making at the design stage and new dynamically adaptable scheduling policies. TrafPy is open-sourced via GitHub and all data associated with this manuscript via RDR.

A Data Set for User Request Trace-Oriented Monitoring and its Applications.

Tracing Processing of Service Requests in Cloud Environments

Visual Analysis of Cloud Computing Performance Using Behavioral Lines

P-Tracer: Path-Based Performance Profiling in Cloud Computing Systems

Workload Characterization on A Cloud Platform: an Early Experience

Mint: Cost-Efficient Tracing with All Requests Collection via Commonality and Variability Analysis

An Online Service-Oriented Performance Profiling Tool for Cloud Computing Systems

Monbench: A Database Performance Benchmark for Cloud Monitoring System

TraceMesh: Scalable and Streaming Sampling for Distributed Traces

Characterizing Job Microarchitectural Profiles at Scale: Dataset and Analysis

Understanding Data Characteristics and Access Patterns in a Cloud Storage System

PerfTrace: A New Multi-metric Network Performance Monitoring Tool

Localizing Root Causes of Performance Anomalies in Cloud Computing Systems by Analyzing Request Trace Logs

SimMon: A Toolkit for Simulating Monitoring Mechanism in Cloud Computing Environments.

Trace-based Multi-Dimensional Root Cause Localization of Performance Issues in Microservice Systems

Transparently Capturing Request Execution Path for Anomaly Detection

TraceTracker: Hardware/Software Co-Evaluation for Large-Scale I/O Workload Reconstruction

Traffic Generation for Benchmarking Data Centre Networks

Capturing Request Execution Path for Understanding Service Behavior and Detecting Anomalies Without Code Instrumentation

BigDataBench: a Big Data Benchmark Suite from Web Search Engines

NetWatch: End-to-End Network Performance Measurement As a Service for Cloud.