Abstract:Remote Procedure Calls (RPC) load balancing is crucial to maximize server resource utilization and reduce tail latency in data center. Existing load balancing methods deploy a frontend server acting as a load balancer, responsible for requesting backend server workload and routing RPC requests to backend server. However, existing methods suffer from two issues. Firstly, they introduce additional latency to RPC by requiring the frontend server to request the resource utilization of backend servers. Secondly, as the number of RPC requests grows, the frontend server becomes a bottleneck in the system, limiting the scalability of the load balancing approach. Programmable switches have become an effective way to serve high traffic rates involving many concurrent flows. In this paper, we propose a novel RPC load balancing approach that enables the programmable switch to take over the RPC request load balancing function. By maintaining a local estimation of the distribution of the backend server workload, the switch can directly guide the selection of a suitable backend server. We formulate optimizing server selection policy as a Bandits with Knapsack (BwK) problem. Our objective is to strike a balance between maximizing the cumulative utilization of server resources and minimizing cumulative latency. Extensive experiments in diverse scenarios demonstrate that the proposed method can effectively improve server selection policy and boost RPC load balancing performance.

An Improved Dynamic Smooth Weighted Round-robin Load-balancing Algorithm

A modified round-robin load-balancing algorithm for cluster-based web servers

Dynamic Load-Balance Model for Web Services

Load Balance Algorithm for Heterogeneous Server Cluster

A Modified Round-Robin Load Balancing Algorithm Based on Content of Request

SDSSE: A Self-Driven RPC Load Balancing Method in Datacenter

An Effective Dynamic Load Balancing Method

Dynamic Load Balancing Algorithm of Distributed File System

A Predictive Modified Round Robin Scheduling Algorithm For Web Server Clusters

A Load Balancing Algorithm Based on Real-Time Performance Dynamic Feedback

DALB: A Dynamic Application-Sensitive Load Balancing Algorithm

Research on Load Balancing Algorithm in Proxy System

Least load-based load balancing strategy for Linux Web cluster

A Content-based Load-balancing Algorithm of Web Cluster

A Dynamic Load Balancing Method with Available Bandwidth Information

Load Balancing in Cloud Computing Environment Using Improved Weighted Round Robin Algorithm for Nonpreemptive Dependent Tasks

A Dynamic Client-Side Load Balancing Mechanism

Dynamic Load Balancing Technology for Cloud-Oriented CDN

An Integrated Routing Scheme with Load Balancing Consideration

A New Type Weights Distribution Scheduling Algorithm with Priority Processing Capability

Implementation of Load Balancing in Distributed Web Server