Abstract:Traditional network resource management mechanisms are mainly flow or packet based. Recently, coflow has been proposed as a new abstraction to capture the communication patterns in a rich set of data parallel applications in data centers. Coflows effectively model the application-level semantics of network resource usage, so high-level optimization goals, such as reducing the transfer latency of applications, can be better achieved by taking coflows as the basic elements in network resource allocation or scheduling. Although efficient coflow scheduling methods have been studied, in this paper, we advocate to schedule weighted coflows as a further step in this direction, where weights are used to express the importances or priorities of different coflows or their corresponding applications. We propose the Weighted Coflow Completion Time (WCCT) minimization problem and a (2 - 2/n+1)-approximate optimal offline algorithm, where n is the concurrent number of coflows. We then design an information-agnostic online algorithm named IAOA to dynamically schedule coflows according to their weights and the instantaneous network condition. We also design and implement a coflow scheduling system named FlyTransfer, which can use the online algorithm as its scheduling method. We test the performance of FlyTransfer by trace-driven simulations as well as real deployment in openstack. Our evaluation results show that, compared to the latest information-agnostic coflow scheduling algorithms, FlyTransfer can reduce more than 40 percent of the WCCT, and more than 30 percent of the completion time for coflows with above-the-average level of importance. It even outperforms the most efficient clairvoyant coflow scheduling method by reducing around 30 percent WCCT, and 25-30 percent of the completion time for coflows with above-the-average importance, respectively.

LAFS: Learning-Based Application-Agnostic Flow Scheduling for Datacenters

An Adaptable and Agnostic Flow Scheduling Approach for Data Center Networks

Rate-aware Flow Scheduling for Commodity Data Center Networks.

Distributed Information-Agnostic Flow Scheduling in Data Centers Based on Wait-Time.

Information-Agnostic Traffic Scheduling in Data Center Networks with Asymmetric Topologies.

Minimizing Datacenter Flow Completion Times with Server-Based Flow Scheduling

Key Flow First Prioritized Flow Scheduling Strategy In Multi-Tenant Data Centers

An Efficient Flow Detection and Scheduling Method in Data Center Networks

Flow-Level Traffic Scheduling in Data Center Networks

PIAS: Practical Information-Agnostic Flow Scheduling for Commodity Data Centers

Efficient Scheduling of Weighted Coflows in Data Centers

Saath: Speeding up CoFlows by Exploiting the Spatial Dimension

Towards Practical and Near-Optimal Coflow Scheduling for Data Center Networks

TAPS: Task-aware preemptive flow scheduling

Fair and Delay Adaptive Scheduler (FDAS) Preliminary Modeling and Optimization

Efficient and Fair: Information-Agnostic Online Coflow Scheduling by Combining Limited Multiplexing with DRL

To schedule or not to schedule: when no-scheduling can beat the best-known flow scheduling algorithm in datacenter networks

Backlog-Aware SRPT Flow Scheduling in Data Center Networks

What Appears Suboptimal May Surprise You: A Fixed-Rate Scheduling Policy for Geo-Distributed CoFlows

Don't be fat: Towards efficient online flow scheduling in data center networks

Towards Stable Flow Scheduling in Data Centers.