Abstract:We present an overview of scalable load balancing algorithms which provide favorable delay performance in large-scale systems, and yet only require minimal implementation overhead. Aimed at a broad audience, the paper starts with an introduction to the basic load balancing scenario, consisting of a single dispatcher where tasks arrive that must immediately be forwarded to one of $N$ single-server queues. A popular class of load balancing algorithms are so-called power-of-$d$ or JSQ($d$) policies, where an incoming task is assigned to a server with the shortest queue among $d$ servers selected uniformly at random. This class includes the Join-the-Shortest-Queue (JSQ) policy as a special case ($d = N$), which has strong stochastic optimality properties and yields a mean waiting time that vanishes as $N$ grows large for any fixed subcritical load. However, a nominal implementation of the JSQ policy involves a prohibitive communication burden in large-scale deployments. In contrast, a random assignment policy ($d = 1$) does not entail any communication overhead, but the mean waiting time remains constant as $N$ grows large for any fixed positive load. In order to examine the fundamental trade-off between performance and implementation overhead, we consider an asymptotic regime where $d(N)$ depends on $N$. We investigate what growth rate of $d(N)$ is required to match the performance of the JSQ policy on fluid and diffusion scale. The results demonstrate that the asymptotics for the JSQ($d(N)$) policy are insensitive to the exact growth rate of $d(N)$, as long as the latter is sufficiently fast, implying that the optimality of the JSQ policy can asymptotically be preserved while dramatically reducing the communication overhead. We additionally show how the communication overhead can be reduced yet further by the so-called Join-the-Idle-Queue scheme, leveraging memory at the dispatcher.

Efficient Multi-Broker Load Balancing in Event Driven Pub-Sub Networks

Operator Placement for Data Stream Processing Based on Publisher/subscriber in Hybrid Cloud-Fog-edge Infrastructure

Efficient Solution for Large-Scale IoT Applications with Proactive Edge-Cloud Publish/Subscribe Brokers Clustering

An efficient content-based notification service routed over p2p network

Topic-based Resource Allocation for Real-Time Publish/subscribe Communication Systems

Modifying Broker Policy for Better Distribution of the Load over Geo-distributed Datacenters

A Survey of Distributed Message Broker Queues

Topic Allocation Method on Edge Servers for Latency-sensitive Notification Service

Network delay-aware load balancing in selfish and cooperative distributed systems

Multi-Objective Optimization of Consumer Group Autoscaling in Message Broker Systems

FogMQ: A Message Broker System for Enabling Distributed, Internet-Scale IoT Applications over Heterogeneous Cloud Platforms

TD-MQTT: Transparent Distributed MQTT Brokers for Horizontal IoT Applications

Efficient Service Broker Policy For Large-Scale Cloud Environments

Towards a Self-Adaptive Middleware for Building Reliable Publish/Subscribe Systems

Modifying Broker Policy for Better Response Time in Datacenters

Highly Scalable and Efficient Publish/Subscribe Protocols Using Geographic Information for Wireless Sensor Networks

Scalable Load Balancing in Networked Systems: Universality Properties and Stochastic Coupling Methods

P2S: a fault-tolerant publish/subscribe infrastructure.

Distributed optimization of event dissemination exploiting interest clustering

Data Load Balancing in Heterogeneous Dynamic Networks

Enhancing the LoRaWAN Publish/Subscribe IoT Data Sharing Model Using Middleman for Smart Grid Application