Abstract:With the popularity of container-based microservices and cloud-native architectures, Kubernetes has established itself as the de facto standard for container orchestration. Kubernetes is known for its advantage in easy deployment and operations for applications; however, it suffers from low resource utilization, incurring high server provisioning and operational costs, due to following reasons. First, it is common practice for latency-sensitive services to be over-provisioned for the peak load: Kubernetes might consider such peak-load provisioning as constant, even though the actual resource utilization is low. Second, the isolation between different containers is poor and cannot prevent performance degradation when best-effort jobs and latency-sensitive services run together. Users may have to run these two classes of workloads on separate machines to avoid interference, resulting in higher provisioning costs. This paper presents a highly scalable cluster scheduling system named Zeus, which is designed based on Kubernetes extension mechanisms. Zeus achieves safe colocation of best-effort jobs and latency-sensitive services. Furthermore, Zeus schedules best-effort jobs based on the real server utilization and can adaptively allocate resources between the two classes of workloads. In addition, Zeus enhances container isolation by coordinating software and hardware isolation features. As a result, Zeus can effectively improve the resource utilization of Kubernetes clusters. We discuss the design and implementation of Zeus and evaluate its effectiveness using latency-sensitive services and best-effort jobs in a massive production environment. The results show that by colocating latency-sensitive services with best-effort jobs, Zeus can increase the average CPU utilization from 15% to 60% without SLO violations.

Revisiting the Design of Mega Data Centers: Considering Heterogeneity Among Containers

Building Mega Data Center from Heterogeneous Containers

MDCube: a high performance network structure for modular data center interconnection.

Zeus: Improving Resource Efficiency Via Workload Colocation for Massive Kubernetes Clusters

Scalable and Cost-Effective Interconnection of Data-Center Servers Using Dual Server Ports

Affinity and Conflict-Aware Placement of Virtual Machines in Heterogeneous Data Centers.

BCube: a high performance, server-centric network architecture for modular data centers

FiConn: Using Backup Port for Server Interconnection in Data Centers

Communication-Aware Container Placement and Reassignment in Large-Scale Internet Data Centers

DDC: A Vision for a Disaggregated Datacenter

Switch Cost and Packet Delay Tradeoff in Data Center Networks with Switch Reconfiguration Overhead

Flexible, highly scalable and cost-effective network structures for data centers

GHDC: a dual-centric data center network architecture by using multi-port servers with greater incremental scalability

Scalability and Bandwidth Optimization for Data Center Networks

Towards the Tradeoffs in Designing Data Center Network Architectures

Application Oriented Dynamic Resource Allocation for Data Centers Using Docker Containers

A Global Cost-Aware Container Scheduling Strategy in Cloud Data Centers

An Energy-Aware Host Resource Management Framework for Two-Tier Virtualized Cloud Data Centers

The stellar transformation: From interconnection networks to datacenter networks

Architecture for Modular Data Centers

Design Of A Load-Balancing Middlebox Based On Sdn For Data Centers