Coflow Scheduling in the Multi-Resource Environment
Jianhui Zhang,Deke Guo,Keqiu Li,Heng Qi,Xiaoyi Tao,Yingwei Jin
DOI: https://doi.org/10.1109/tnsm.2019.2901549
2019-06-01
IEEE Transactions on Network and Service Management
Abstract:In data centers, a lot of cluster computing applications follow the coflow working pattern. That is, a collection of flows between two groups of machines is semantically related. On the other hand, network function virtualization sufficiently improves the performance of data center networks. It however complicates the network environment by introducing many multi-function middleboxes each with multiple resources. Coflows encounter extremely different processing delays under diverse network functions. Prior coflow scheduling schemes are insufficient to guarantee the coflow completion time in the multi-resource environment. In this paper, we propose, model, and analyze the coflow scheduling problem in the multi-resource environment. We present a dedicated method, data rate guarantee for coflow (DRGC), to guarantee the data rate requirements of coflows in this situation. DRGC prioritizes the coflow scheduling sequence, assigns precise data rates for coflows, and deploys a packet scheduling algorithm at middleboxes to guarantee their transmissions. In our experiments, DRGC efficiently guarantees the completion times of coflows and supports 15% more workload, compared with other scheduling schemes.
computer science, information systems