Virtual Cluster Based Resource Sharing Approach for Private Cloud Environment
Peidong LIU,Bo AN,Yehong ZHONG,Hu WANG,Donggang CAO
DOI: https://doi.org/10.3778/j.issn.1673-9418.1609030
2017-01-01
Abstract:With the rapid development of cloud computing and big data processing, an increasing number of applica-tion frameworks are being considered to run in a"cloud way". This development brings about several challenges to the enterprise private cloud computing platform, e.g., sharing limited cluster resources effectively among different frameworks and users, being able to run most existing heterogeneous applications, as well as providing scalability and elasticity support for newly emerged frameworks. This paper proposes a new service model, namely, cluster as a service (ClaaS), which is suitable for medium and small-sized data centers to solve these problems in a relatively easy and general way. The idea behind this model is to virtualize the cluster environment for distributed application frameworks. Most applications can directly run in the virtual cluster environment without any modification, which is a great advantage. Based on lightweight containers and software-defined network, this paper implements an open-source system of ClaaS named Docklet to prove the feasibility of this service model. Docklet is able to distribute resources to users according to their needs, support frameworks like Spark, MPI, and languages like Python, R and Java. Moreover, Docklet is now used to provide and deploy services in real scenarios of teaching and researching. Finally, this paper describes the design and implementation of this system, then presents several examples and evalu-ates the entire system.