Abstract:Recently, the large-scale cluster of data center is usually constructed to support both HPC and Cloud computing. It can be explained from two aspects: (1) The data center is typically a sharing environment for all the users, users may submit different types of jobs (HPC and Cloud computing) for processing currently; (2) Some applications can be divided into two parts of subtasks which are suitable to HPC and Cloud computing respectively, e.g. the AMS (Alpha Magnetic Spectrometer) experiment is such a typical application. Thus in order to provide good service for both computing models, it is needed to construct a HPC and Cloud hybrid environment. An existing management mechanism is to allocate fixed proportions of resources for different application environments. However, this approach has a significant performance drawback that is the low resource utilization. In order to overcome this drawback, we propose a dynamic resource management framework and mechanism to satisfy the requirements of both HPC and Cloud computing. Firstly we present a prediction model that is used to predict the arrival rate of all kinds of jobs (HPC types and Cloud types). Based on the prediction results, we propose a dynamic resource allocation algorithm, which manages dynamic resources allocation by using queuing theory. Finally, we evaluate our mechanism by real data sets from AMS experiment and Cloud tasks running on the HPC center in Southeast University. The results show that the proposed mechanism can effectively improve resource utilization at least 30% in this hybrid environment.

Providing a Cloud Infrastructure on Clusters

A Cloud Computing Infrastructure on Heterogeneous Computing Resources.

A Novel Job Scheduling Model to Enhance Efficiency and Overall User Fairness of Cloud Computing Environment.

Cloud Storage As the Infrastructure of Cloud Computing

Design and Implementation of a High Availability Cluster Based on Cloud Computing

Cloud Versus In-House Cluster

An Autonomic Capacity Management Approach with Cloud Insight Towards Cost-Efficient Throughput Optimization for High Performance Computing

A Cloud Computing Management System Platform Based on Power-Sensitive Resource

An Efficient Resource Management System for On-Line Virtual Cluster Provision

Towards Providing Cloud Functionalities for Grid Users

Virtual Cluster Based Resource Sharing Approach for Private Cloud Environment

Cluster As a Service: A Resource Sharing Approach for Private Cloud

Cluster As a Service: A Container Based Cluster Sharing Approach with Multi-user Support

Modelling, Evaluating and Designing Virtual Machine Scheduling by a Clustering Mechanism in Cloud Computing Environments.

Efficient Resource Management for Cloud Computing Environments

Cloud Scheduler: a resource manager for distributed compute clouds

Cluster Scheduler on Heterogeneous Cloud

Dynamic Resource Management in a HPC and Cloud Hybrid Environment

A New Scheme for Improving the Utilisation of Nested-Cloud Resources

The feasibility and properties of dividing virtual machine resources using the virtual machine cluster as the unit in cloud computing