Abstract:To achieve a high degree of resource utilization, production clusters need to co-schedule diverse workloads – including both batch analytic jobs with short-lived tasks and long-running applications (LRAs) that execute for a long time frame from hours to months – onto the shared resources. Microservice architecture advances the manifestation of distributed LRAs (DLRAs), comprising multiple interconnected microservices that are executed in long-lived distributed containers and serve massive user requests. Detecting and mitigating QoS violation become even more intractable due to the network uncertainties and latency propagation across dependent microservices. However, current resource managers are only responsible for resource allocation among applications/jobs but agnostic to runtime QoS such as latency at application level. The state-of-the-art QoS-aware scheduling approaches are dedicated for monolithic applications, without considering the temporal-spatio performance variability across distributed microservices. In this paper, we present Toposch , a new scheduling and execution framework to prioritize the QoS of DLRAs whilst balancing the performance of batch jobs and maintaining high cluster utilization through harvesting idle resources. Toposch tracks footprints of every single request across microservices and uses critical path analysis, based on the end-to-end latency graph, to identify microservices that have high risk of QoS violation. Based on microservice and node level risk assessment, we intervene the batch scheduling by adaptively reducing the visible resources to batch tasks and thus delaying their execution to give way to DLRAs. We propose a prediction-based vertical resource auto-scaling mechanism, with the aid of resource-performance modeling and fine-grained resource inference and access control, for prompt recovery of QoS violation. A cost-effective task preemption is leveraged to ensure a low-cost task preemption and resource reclamation during the auto-scaling. Toposch is integrated with Apache YARN and experiments show that Toposch outperforms other baselines in terms of performance guarantee of DLRAs, at an acceptable cost of batch job slowdown. The tail latency of DLRAs is merely 1.12x of the case of executing alone on average in Toposch with a 26% JCT increase of Spark analytic jobs.

Nodens: Enabling Resource Efficient and Fast QoS Recovery of Dynamic Microservice Applications in Datacenters

Microservice Auto-Scaling Algorithm Based on Workload Prediction in Cloud-Edge Collaboration Environment

Achieving Real-Time Quality of Service with Software Defined Networking

Qos-Aware Indiscriminate Volume Storage Cloud

Dynamically Adjusting Scale of a Kubernetes Cluster under QoS Guarantee

QoS-Aware and Resource Efficient Microservice Deployment in Cloud-Edge Continuum

Adaptive Resource Efficient Microservice Deployment in Cloud-Edge Continuum

Adaptive QoS-aware Microservice Deployment with Excessive Loads Via Intra- and Inter-Datacenter Scheduling

QoS-awareness of Microservices with Excessive Loads Via Inter-Datacenter Scheduling

Dynamic Economic- And Qos-Based Approach For Workflow-Oriented Distributed Services Allocation

QoS-Aware Co-Scheduling for Distributed Long-Running Applications on Shared Clusters.

Topology-Aware Scheduling Framework for Microservice Applications in Cloud

Design Of A Load-Balancing Middlebox Based On Sdn For Data Centers

Dynamically Balancing Load with Overload Control for Microservices

On the Joint Design of Microservice Deployment and Routing in Cloud Data Centers

Aggressive Resource Provisioning for Ensuring QoS in Virtualized Environments

Joint Resource Provisioning for Internet Datacenters with Diverse and Dynamic Traffic

Sinan: Data Driven Resource Management for Cloud Microservices

Service Decoupling and Deployment Strategy for Edge Computing

QaMeC: A QoS-driven IoVs Application Optimizing Deployment Scheme in Multimedia Edge Clouds

Ephedrine QoS: an Antidote to Slow, Congested, Bufferless NoCs