Abstract:Due to the intensive computation requirements of emerging applications and the limited computational capability of edge computing servers, the computation task must be executed on multiple edge servers in a distributive and cooperative manner. However, the large amount of information exchanged among the edge servers is a major obstacle for improving the computing performance. By utilizing the excess computational resource, coded MapReduce provides an effective approach to reduce the communication load. In this paper, we develop a stochastic load scheduling framework to complete the computation tasks with coded MapReduce considering the intrinsic tradeoff between the communication and computation loads. Our goal is to minimize the communication load under time-varying excess computational resources. We first reduce this problem to a task scheduling problem by exploiting the property of the computing repetition in the coded MapReduce framework. Since the task scheduling problem is still a stochastic optimization problem, it is generally difficult to solve. In the offline setting, we obtain the optimal computation load scheduling algorithm by adopting the augmented Lagrangian method. In the online setting, we derive a worst-case performance bound of the online equal task scheduling (ETS) algorithm by using competitive analysis. Furthermore, we make full use of past state information of computing resources for pre-planing and propose an improved algorithm based on the ETS algorithm in a learning manner. Finally, our proposed algorithm is evaluated by simulation to demonstrate that the proposed algorithms are superior over the conventional algorithms, and the performance gap between the online and offline algorithms is fairly small.

Adaptive Distributed Load Scheduling Strategy

Distributed Model Based on Data Partition and Load Balance Algorithm

An Efficient Fault-Tolerant Scheduling Algorithm for Periodic Real-Time Tasks in Heterogeneous Platforms

SDSSE: A Self-Driven RPC Load Balancing Method in Datacenter

Intelligent Agent-Based Scheduling Mechanism for Grid Service

Strategy and simulation of adaptive RID for distributed dynamic load balancing in parallel systems

DALB: A Dynamic Application-Sensitive Load Balancing Algorithm

Adaptive Dual-Threshold Dynamic Load Balancing System Based on Migrating

Asynchronous Distributed Optimal Load Scheduling Algorithm

Dynamic Load Balancing Algorithm of Distributed File System

A Decentralized Load Balancing Architecture for Cache System

A Dynamic Client-Side Load Balancing Mechanism

Adaptive bidding load balancing algorithms in heterogeneous distributed systems

An adaptive load balancing framework for parallel database systems based on collaborative agents

Integrated Feedback Scheduling Of Networked Control Systems

Dynamic Scheduling Algorithm for Local Interconnect Network and Its Application

Load scheduling for distributed edge computing: A communication-computation tradeoff

Dynamic Feedback Scheduling Algorithm in LVS

DLBS: Decentralized load balancing scheme for event-driven cloud frameworks

Load Balancing in Distributed Workflow Management System

An Event-Driven Dynamic Load Balancing Strategy For Streaming Media Clustered Server Systems