Abstract:To achieve a high degree of resource utilization, production clusters need to co-schedule diverse workloads – including both batch analytic jobs with short-lived tasks and long-running applications (LRAs) that execute for a long time frame from hours to months – onto the shared resources. Microservice architecture advances the manifestation of distributed LRAs (DLRAs), comprising multiple interconnected microservices that are executed in long-lived distributed containers and serve massive user requests. Detecting and mitigating QoS violation become even more intractable due to the network uncertainties and latency propagation across dependent microservices. However, current resource managers are only responsible for resource allocation among applications/jobs but agnostic to runtime QoS such as latency at application level. The state-of-the-art QoS-aware scheduling approaches are dedicated for monolithic applications, without considering the temporal-spatio performance variability across distributed microservices. In this paper, we present Toposch , a new scheduling and execution framework to prioritize the QoS of DLRAs whilst balancing the performance of batch jobs and maintaining high cluster utilization through harvesting idle resources. Toposch tracks footprints of every single request across microservices and uses critical path analysis, based on the end-to-end latency graph, to identify microservices that have high risk of QoS violation. Based on microservice and node level risk assessment, we intervene the batch scheduling by adaptively reducing the visible resources to batch tasks and thus delaying their execution to give way to DLRAs. We propose a prediction-based vertical resource auto-scaling mechanism, with the aid of resource-performance modeling and fine-grained resource inference and access control, for prompt recovery of QoS violation. A cost-effective task preemption is leveraged to ensure a low-cost task preemption and resource reclamation during the auto-scaling. Toposch is integrated with Apache YARN and experiments show that Toposch outperforms other baselines in terms of performance guarantee of DLRAs, at an acceptable cost of batch job slowdown. The tail latency of DLRAs is merely 1.12x of the case of executing alone on average in Toposch with a 26% JCT increase of Spark analytic jobs.

ROSE: Cluster Resource Scheduling Via Speculative Over-Subscription

Performance-Aware Speculative Resource Oversubscription for Large-Scale Clusters

A Novel Job Scheduling Model to Enhance Efficiency and Overall User Fairness of Cloud Computing Environment.

Local Adaptive Resource Scheduling for Internet-Based Computation on SMT Platform

Multi-Tiered On-Demand Resource Scheduling for VM-Based Data Center

Not All Resources are Visible: Exploiting Fragmented Shadow Resources in Shared-State Scheduler Architecture

An Integrated Dynamic Resource Scheduling Framework in On-Demand Clouds.

An Integrated Dynamic Resource Scheduling Framework in On-Demand Clouds

A User-Level NUMA-Aware Scheduler for Optimizing Virtual Machine Performance.

Improving Cluster Resource Efficiency with Oversubscription

Fine-grained Scheduling in Multi-Resource Clusters

Cluster Scheduler on Heterogeneous Cloud

Sonnet: A control-theoretic approach for resource allocation in cluster management

An Optimized Resource Scheduling Strategy for Hadoop Speculative Execution Based on Non-cooperative Game Schemes

Improving resource utilization by timely fine-grained scheduling

A HPC Co-Scheduler with Reinforcement Learning

QoS-Aware Co-Scheduling for Distributed Long-Running Applications on Shared Clusters.

Methods of Resource Scheduling Based on Optimized Fuzzy Clustering in Fog Computing

DRESS: Dynamic RESource-Reservation Scheme for Congested Data-Intensive Computing Platforms

Ras: A Task Scheduling Algorithm Based On Resource Attribute Selection In A Task Scheduling Framework

An Efficient Two-Level Hierarchy Job Scheduling and Task Dispatching Strategy for Cluster Rendering System