Abstract:Container-based virtualization techniques are becoming an alternative to traditional virtual machines, due to less overhead and better scaling. As one of the most widely used open-source container orchestration systems, Kubernetes provides a built-in mechanism, that is, horizontal pod autoscaler (HPA), for dynamic resource provisioning. By default, scaling pods only based on CPU utilization, a single performance metric, HPA may create more pods than actually needed. Through extensive measurements of a containerized n-tier application benchmark, RUBBoS, we find that excessive pods consume more CPU and memory and even deteriorate response times of applications, due to interference. Furthermore, a Kubernetes service does not balance incoming requests among old pods and new pods created by HPA, due to stateful HTTP. In this paper, we propose a bi-metric approach to scaling pods by taking into account both CPU utilization and utilization of a thread pool, which is a kind of important soft resource in Httpd and Tomcat. Our approach collects the utilization of CPU and memory of pods. Meanwhile, it makes use of ELBA, a milli-bottleneck detector, to calculate queue lengths of Httpd and Tomcat pods and then evaluate the utilization of their thread pools. Based on the utilization of both CPU and thread pools, our approach could scale up less replicas of Httpd and Tomcat pods, contributing to a reduction of hardware resource utilization. At the same time, our approach leverages preStop hook along with liveness and readiness probes to relieve load imbalance among old Tomcat pods and new ones. Based on the containerized RUBBoS, our experimental results show that the proposed approach could not only reduce the usage of CPU and memory by as much as 14% and 24% when compared with HPA, but also relieve the load imbalance to reduce average response time of requests by as much as 80%. Our approach also demonstrates that it is better to scale pods by multiple metrics rather than a single one.

High Concurrency Response Strategy based on Kubernetes Horizontal Pod Autoscaler

Dynamically Adjusting Scale of a Kubernetes Cluster under QoS Guarantee

Microservice Auto-Scaling Algorithm Based on Workload Prediction in Cloud-Edge Collaboration Environment

HCA Operator: A Hybrid Cloud Auto-scaling Tooling for Microservice Workloads.

A Trend Detection-Based Auto-Scaling Method for Containers in High-Concurrency Scenarios

Zeus: Improving Resource Efficiency Via Workload Colocation for Massive Kubernetes Clusters

Online Cost-Aware Service Requests Scheduling in Hybrid Clouds for Cloud Bursting

AHPA: Adaptive Horizontal Pod Autoscaling Systems on Alibaba Cloud Container Service for Kubernetes

Online Cost-Rejection Rate Scheduling for Resource Requests in Hybrid Clouds

Hybrid Autoscaling Strategy on Container-Based Cloud Platform

LP-HPA: Load Predict-Horizontal Pod Autoscaler for Container Elastic Scaling.

Smart HPA: A Resource-Efficient Horizontal Pod Auto-scaler for Microservice Architectures

A Bi-Metric Autoscaling Approach for N-Tier Web Applications on Kubernetes

Horizontal Pod Autoscaling in Kubernetes for Elastic Container Orchestration

Hybrid Cloud Adaptive Scheduling Strategy for Heterogeneous Workloads

Auto-scaling HTCondor pools using Kubernetes compute resources

A Cost-Efficient Container Orchestration Strategy in Kubernetes-Based Cloud Computing Infrastructures with Heterogeneous Resources

On the Optimization of Kubernetes toward the Enhancement of Cloud Computing

HANSEL: Adaptive Horizontal Scaling of Microservices Using Bi-LSTM

SLA-Adaptive Threshold Adjustment for a Kubernetes Horizontal Pod Autoscaler