Abstract:The transition from monolithic architecture to microservices has enhanced flexibility in application design and its scalable execution. This approach often involves using a computing cluster managed by a container orchestration platform, which supports the deployment of microservices. However, this shift introduces significant challenges, particularly in the efficient scheduling of containerized services. These challenges are compounded by unpredictable scenarios such as dynamic incoming workloads with various execution traffic and variable communication delays among cluster nodes. Existing works often overlook the real-time traffic impacts of dynamic requests on running microservices, as well as the varied communication delays across cluster nodes. Consequently, even optimally deployed microservices could suffer from significant performance degradation over time. To address these issues, we introduce a network and traffic-aware adaptive scheduling framework, TraDE. This framework can adaptively redeploy microservice containers to maintain desired performance amid changing traffic and network conditions within the hosting cluster. We have implemented TraDE as an extension to the Kubernetes platform. Additionally, we deployed realistic microservice applications in a real compute cluster and conducted extensive experiments to assess our framework's performance in various scenarios. The results demonstrate the effectiveness of TraDE in rescheduling running microservices to enhance end-to-end performance while maintaining a high goodput ratio. Compared with the existing method NetMARKS, TraDE outperforms it by reducing the average response time of the application by up to 48.3\%, and improving the throughput by up to 1.4x while maintaining a goodput ratio of 95.36\% and showing robust adaptive capability under sustained workloads.
Networking and Internet Architecture,Distributed, Parallel, and Cluster Computing,Emerging Technologies,Performance
What problem does this paper attempt to address?
### What problems does this paper attempt to solve?
This paper aims to solve the scheduling challenges faced by the microservice architecture in dynamic computing environments, especially when dealing with unpredictable workloads and changes in cross - node communication latency. Specifically:
1. **Background of the microservice architecture**:
- With the transition from monolithic architecture to microservice architecture, although the flexibility and scalability of application design are enhanced, new challenges are also introduced, especially in the effective scheduling of containerized services.
- Dynamic workloads, changes in execution traffic, and changes in communication latency between cluster nodes make it difficult to ensure the performance of microservices.
2. **Limitations of existing methods**:
- Existing methods usually ignore the impact of real - time traffic on running microservices and the changing communication latency between different nodes.
- These factors lead to significant performance degradation over time even for optimally deployed microservices.
3. **Proposed problems**:
- How to optimize the deployment of microservice instances on different server nodes in a dynamic network and traffic environment to improve the overall end - to - end performance of the application?
- In particular, when dealing with QoS (Quality of Service) violations, how to adaptively reschedule microservice containers to maintain the expected performance goals?
4. **Specific problem definition**:
- The paper formalizes the problem as a **service - node mapping problem**, that is, in an optimization problem, the goal is to map a set of microservices to a set of server nodes to minimize certain performance metrics (such as communication latency and overall response time).
- Given a set of microservices \( M=\{m_1,m_2,\ldots,m_k\} \), a set of servers \( N = \{n_1,n_2,\ldots,n_p\} \), a traffic matrix \( \text{Traffic} \), where \( \text{Traffic}[i][j] \) represents the bidirectional traffic between microservices \( m_i \) and \( m_j \), and a delay matrix \( \text{Delay} \), where \( \text{Delay}[i][j] \) represents the communication latency between server nodes \( n_i \) and \( n_j \), the goal is to find a mapping \( P:M\rightarrow N \) such that the total communication cost is minimized:
\[
\text{Total Cost}=\sum_{i = 1}^{k}\sum_{j = 1}^{k}\text{Traffic}[i][j]\cdot\text{Delay}[P(m_i)][P(m_j)]
\]
- At the same time, resource constraints need to be satisfied:
\[
\sum_{m_i\in P^{-1}(n_j)}R_i\leq C_j\quad\forall n_j\in N
\]
where \( R_i \) is the amount of resources required by microservice \( m_i \), and \( C_j \) is the capacity of server \( n_j \).
By solving these problems, the paper proposes a framework named **TraDE**, which can adaptively redeploy microservice containers to cope with dynamic requests and changes in cross - node communication latency, thereby maintaining or improving the performance of the application.