Abstract:Microservice architectures (MSAs) can increase the performance of distributed systems and enable better resource allocation by sharing underlying resources among multiple microservices (MSs). One of the main advantages of MSAs is the ability to leverage the elasticity provided by an infrastructure so that only the most demanding services are scaled, which can contribute to efficient allocation of processing resources. A major problem in allocating resources to microservices is determining a set of auto-scaling parameters that will result in all microservices meeting specific service level agreements (SLAs). Since the space of feasible configurations can be vast, manually determining a combination of parameter values that will result in all SLAs being met is complex and time consuming. In addition, the performance overhead caused by running microservices concurrently and the overhead caused by the VM instantiation process must also be evaluated. Another problem is that microservices can suffer performance degradation due to resource contention, which depends on how microservices are distributed across servers. To address the aforementioned issues, this paper proposes the modeling of these infrastructures and their auto-scaling mechanisms in a private cloud using stochastic Petri nets (SPNs), the non-dominated sorting genetic algorithm II (NSGA-II), one of the most popular evolutionary algorithms for multiobjective optimization (MOO), and random forest regression (RFR), an ensemble-learning-based method, to identify critical trade-offs between performance and resource consumption considering all deployed MSs. The SPN-based model is capable of representing both instantiation of elastic VMs and a pool of instantiated elastic VMs where only containers are started. The analytical framework enables service providers (SPs) to estimate performance metrics considering configurations that satisfy all performance constraints, use of elastic VMs, discard rate, discard probability, throughput, response time, and corresponding cumulative distribution functions (CDFs). These metrics are critical because they make it possible to estimate the time required to process each request, the number of requests processed in a time interval, the number of requests rejected, and the utilization of resources. The framework was validated with 95% confidence interval (CI) using a real-world testbed. Two case studies were used to investigate its feasibility by evaluating its application in a real scenario. We noticed a significant improvement in performance when using a pool of elastic VMs, where throughput improved by 21.5% and the number of discarded requests decreased by 70%. The application of the framework can help in finding optimized solutions that support both infrastructure planning and online performance prediction, and enable trade-off analyses considering different scenarios and constraints.

AlphaR: Learning-Powered Resource Management for Irregular, Dynamic Microservice Graph.

Microservice Auto-Scaling Algorithm Based on Workload Prediction in Cloud-Edge Collaboration Environment

MSARS: A Meta-Learning and Reinforcement Learning Framework for SLO Resource Allocation and Adaptive Scaling for Microservices

Efficient Microservice Deployment in the Edge-Cloud Networks With Policy-Gradient Reinforcement Learning

SRAF: A Service-Aware Resource Allocation Framework for VM Management in Mobile Data Networks

Practice of Alibaba Cloud on Elastic Resource Provisioning for Large-scale Microservices Cluster

Runtime Resource Management for Microservices-Based Applications: A Congestion Game Approach (Short Paper)

DRPC: Distributed Reinforcement Learning Approach for Scalable Resource Provisioning in Container-based Clusters

Digital Twin-assisted Reinforcement Learning for Resource-aware Microservice Offloading in Edge Computing

Astraea: towards QoS-aware and resource-efficient multi-stage GPU services

Analytically-Driven Resource Management for Cloud-Native Microservices

Intelligent Hybrid Resource Allocation in MEC-assisted RAN Slicing Network

DyPS: Dynamic Parameter Sharing in Multi-Agent Reinforcement Learning for Spatio-Temporal Resource Allocation

Joint Service Migration and Resource Allocation in Edge IoT System Based on Deep Reinforcement Learning

Graph Neural Network-Based SLO-Aware Proactive Resource Autoscaling Framework for Microservices

Deep Reinforcement Learning-Based Service-Oriented Resource Allocation in Smart Grids

A2C-DRL: Dynamic Scheduling for Stochastic Edge-Cloud Environments Using A2C and Deep Reinforcement Learning

A performance modeling framework for microservices-based cloud infrastructures

Dynamic Public Resource Allocation Based on Human Mobility Prediction

Microservice Selection in Edge-Cloud Collaborative Environment: A Deep Reinforcement Learning Approach

LSRAM: A Lightweight Autoscaling and SLO Resource Allocation Framework for Microservices Based on Gradient Descent