A Resource Co-Allocation Method for Load-Balance Scheduling over Big Data Platforms

Wanchun Dou,Xiaolong Xu,Xiang Liu,Laurence T. Yang,Yiping Wen
DOI: https://doi.org/10.1016/j.future.2017.07.009
IF: 7.307
2018-01-01
Future Generation Computer Systems
Abstract:Recently, an increasingly large number of high-performance computing (HPC) applications have been developed and hosted in clouds. The cloud computing scheme provides a model of utility computing, which is often implemented in the form of big data centers. As HPC applications are usually resource-intensive, how to exploit economies of scale to harmonize the resource allocation among large-scale HPC applications under performance constraints often poses a significant challenge on the load-balance scheduling policies in big data platforms. In view of this challenge, in this paper we propose a Resource C o-Allocation method, named RCA, for load-balance scheduling. Technically, the method is promoted by four steps that are enacted in a hierarchical control framework. The effectiveness and efficiency of the proposed method are validated by extensive experiments.
What problem does this paper attempt to address?