Abstract:Remote Procedure Calls (RPC) load balancing is crucial to maximize server resource utilization and reduce tail latency in data center. Existing load balancing methods deploy a frontend server acting as a load balancer, responsible for requesting backend server workload and routing RPC requests to backend server. However, existing methods suffer from two issues. Firstly, they introduce additional latency to RPC by requiring the frontend server to request the resource utilization of backend servers. Secondly, as the number of RPC requests grows, the frontend server becomes a bottleneck in the system, limiting the scalability of the load balancing approach. Programmable switches have become an effective way to serve high traffic rates involving many concurrent flows. In this paper, we propose a novel RPC load balancing approach that enables the programmable switch to take over the RPC request load balancing function. By maintaining a local estimation of the distribution of the backend server workload, the switch can directly guide the selection of a suitable backend server. We formulate optimizing server selection policy as a Bandits with Knapsack (BwK) problem. Our objective is to strike a balance between maximizing the cumulative utilization of server resources and minimizing cumulative latency. Extensive experiments in diverse scenarios demonstrate that the proposed method can effectively improve server selection policy and boost RPC load balancing performance.

Double-dispatcher-based Load Balancing Solution on Web Service Cluster

Dynamic Load-Balance Model for Web Services

Distributed Model Based on Data Partition and Load Balance Algorithm

Request Dispatching Algorithms for Web Server Clusters Based on Load Balancing

Fuzzy Clustering with Feature Weight Preferences for Load Balancing in Cloud

SDSSE: A Self-Driven RPC Load Balancing Method in Datacenter

A Load Balancing Strategy Based on Data Correlation in Cloud Computing

Informed Live Migration Strategies Of Virtual Machines For Cluster Load Balancing

Design of High Availability Load Balancing Information Publishing Platform

Performance Evaluation of QoS-Aware Load Balancing of Web-server Clusters

Research on request dispatching algorithm for Web server clusters

Study and Implementation of Grid Service Server Performance Optimization System

A Switching Difference Control Method for Load Balancing Control of Web-server Clusters

Load Balance Algorithm for Heterogeneous Server Cluster

A modified round-robin load-balancing algorithm for cluster-based web servers

Load Balancing Algorithm for Middleware Database Cluster

Multi-Cluster Load Balancing Based On Process Migration

A Dynamic Client-Side Load Balancing Mechanism

Modeling And Performance Analysis Of Qos-Aware Load Balancing Of Web-Server Clusters

A Policy of Task Allocation Base on Distributed Cluster Computing Towards Cloud

Analysis of Load-Balancing of Web Cluster Based on Tls Session Sharing