Abstract:To achieve a high degree of resource utilization, production clusters need to co-schedule diverse workloads – including both batch analytic jobs with short-lived tasks and long-running applications (LRAs) that execute for a long time frame from hours to months – onto the shared resources. Microservice architecture advances the manifestation of distributed LRAs (DLRAs), comprising multiple interconnected microservices that are executed in long-lived distributed containers and serve massive user requests. Detecting and mitigating QoS violation become even more intractable due to the network uncertainties and latency propagation across dependent microservices. However, current resource managers are only responsible for resource allocation among applications/jobs but agnostic to runtime QoS such as latency at application level. The state-of-the-art QoS-aware scheduling approaches are dedicated for monolithic applications, without considering the temporal-spatio performance variability across distributed microservices. In this paper, we present Toposch , a new scheduling and execution framework to prioritize the QoS of DLRAs whilst balancing the performance of batch jobs and maintaining high cluster utilization through harvesting idle resources. Toposch tracks footprints of every single request across microservices and uses critical path analysis, based on the end-to-end latency graph, to identify microservices that have high risk of QoS violation. Based on microservice and node level risk assessment, we intervene the batch scheduling by adaptively reducing the visible resources to batch tasks and thus delaying their execution to give way to DLRAs. We propose a prediction-based vertical resource auto-scaling mechanism, with the aid of resource-performance modeling and fine-grained resource inference and access control, for prompt recovery of QoS violation. A cost-effective task preemption is leveraged to ensure a low-cost task preemption and resource reclamation during the auto-scaling. Toposch is integrated with Apache YARN and experiments show that Toposch outperforms other baselines in terms of performance guarantee of DLRAs, at an acceptable cost of batch job slowdown. The tail latency of DLRAs is merely 1.12x of the case of executing alone on average in Toposch with a 26% JCT increase of Spark analytic jobs.

Sonnet: A control-theoretic approach for resource allocation in cluster management

Dynamically Adjusting Scale of a Kubernetes Cluster under QoS Guarantee

Time-sharing Parallel Applications Through Performance-Targeted Feedback-Controlled Real-Time Scheduling.

ROSE: Cluster Resource Scheduling Via Speculative Over-Subscription

Performance-Aware Speculative Resource Oversubscription for Large-Scale Clusters

Improving resource utilization by timely fine-grained scheduling

QoS-Aware Co-Scheduling for Distributed Long-Running Applications on Shared Clusters.

Sinan: Data Driven Resource Management for Cloud Microservices

A Novel Resource Scheduling Approach in Container Based Clouds

A Cost-Efficient Container Orchestration Strategy in Kubernetes-Based Cloud Computing Infrastructures with Heterogeneous Resources

Optimizing Cloud Performance: A Microservice Scheduling Strategy for Enhanced Fault-Tolerance, Reduced Network Traffic, and Lower Latency

A new container scheduling algorithm based on multi-objective optimization

Task rescheduling optimization to minimize network resource consumption

On Global Resource Allocation in Clusters for Data Analytics.

Improving Cluster Resource Efficiency with Oversubscription

Cluster Scheduler on Heterogeneous Cloud

Optimal Resource Allocation Using Genetic Algorithm in Container-Based Heterogeneous Cloud

Towards Enhanced Energy Aware Resource Optimization for Edge Devices Through Multi-cluster Communication Systems

Economical and Robust Provisioning of N-Tier Cloud Workloads: A Multi-level Control Approach

Self-adaptive Resource Management for Large-Scale Shared Clusters

EdgeOptimizer : A programmable containerized scheduler of time-critical tasks in Kubernetes -based edge-cloud clusters