Abstract:Containerization has been used in many applications for isolation purposes due to its lightweight, scalable, and highly portable properties. However, to apply containerization in large-scale Internet data centers faces a big challenge. Services in data centers are always instantiated as a group of containers, which often generate heavy communication workloads and therefore resulting in inefficient communications and downgraded service performance. Although assigning the containers of the same service to the same server can reduce the communication overhead, this may cause heavily imbalanced resource utilization since containers of the same service are usually intensive to the same resource. To reduce communication cost as well as balance the resource utilization in large-scale data centers, we further explore the container distribution issues in a real industrial environment and find that such conflict lies in two phases-container placement and container reassignment. The objective of this paper is to address the container distribution problem in these two phases. For the container placement problem, we propose an efficient communication aware worst fit decreasing algorithm to place a set of new containers into data centers. For the container reassignment problem, we propose a two-stage algorithm called Sweep&Search to optimize a given initial distribution of containers by migrating containers among servers. We implement the proposed algorithms in Baidu's data centers and conduct extensive evaluations. Compared with the state-of-the- art strategies, the evaluation results show that our algorithms perform better up to 70% and increase the overall service throughput up to 90% simultaneously.

Data Allocation Algorithm Based on Visit Capacity and Dependency Evaluation in Cloud

ON TRAFFIC MANAGEMENT OF ONLINE CLOUD STORAGE SERVICE

A Load Balancing Strategy Based on Data Correlation in Cloud Computing

Distributed Cache Memory Data Migration Strategy Based on Cloud Computing

Towards efficient load distribution in big data cloud

Location-Aware Data Block Allocation Strategy for HDFS-Based Applications in the Cloud

A Policy of Task Allocation Base on Distributed Cluster Computing Towards Cloud

On Efficient Bandwidth Allocation for Traffic Variability in Datacenters

Efficient Data Cache Allocation for Sensor Clouds

A Resource Co-Allocation Method for Load-Balance Scheduling over Big Data Platforms

Cloud Loading Balance Algorithm

SSUR: An Approach to Optimizing Virtual Machine Allocation Strategy Based on User Requirements for Cloud Data Center

An Intelligent Optimization Method for Optimal Virtual Machine Allocation in Cloud Data Centers

Research on Data Storage Technology in Cloud Computing Environment

Dynamic system allocation and application of cloud computing virtual resources based on system architecture

Optimizing Multi-Cloud CDN Deployment and Scheduling Strategies Using Big Data Analysis

A Cost-Effective Strategy for Storing Scientific Datasets with Multiple Service Providers in the Cloud

SEA: Stable resource allocation in geographically distributed clouds

An Algorithm for Network and Data-aware Placement of Multi-Tier Applications in Cloud Data Centers

Communication-Aware Container Placement and Reassignment in Large-Scale Internet Data Centers

Network Bandwidth Allocation Problem For Cloud Computing