Abstract:Due to the intensive computation requirements of emerging applications and the limited computational capability of edge computing servers, the computation task must be executed on multiple edge servers in a distributive and cooperative manner. However, the large amount of information exchanged among the edge servers is a major obstacle for improving the computing performance. By utilizing the excess computational resource, coded MapReduce provides an effective approach to reduce the communication load. In this paper, we develop a stochastic load scheduling framework to complete the computation tasks with coded MapReduce considering the intrinsic tradeoff between the communication and computation loads. Our goal is to minimize the communication load under time-varying excess computational resources. We first reduce this problem to a task scheduling problem by exploiting the property of the computing repetition in the coded MapReduce framework. Since the task scheduling problem is still a stochastic optimization problem, it is generally difficult to solve. In the offline setting, we obtain the optimal computation load scheduling algorithm by adopting the augmented Lagrangian method. In the online setting, we derive a worst-case performance bound of the online equal task scheduling (ETS) algorithm by using competitive analysis. Furthermore, we make full use of past state information of computing resources for pre-planing and propose an improved algorithm based on the ETS algorithm in a learning manner. Finally, our proposed algorithm is evaluated by simulation to demonstrate that the proposed algorithms are superior over the conventional algorithms, and the performance gap between the online and offline algorithms is fairly small.

An Expanded Stochastic Model in DAG Schemes

Runtime Parallel Incremental Scheduling of DAGs

A Runtime System for Dynamic DAG Programming

Optimization of uncertain dependent task mapping on heterogeneous computing platforms

Load scheduling for distributed edge computing: A communication-computation tradeoff

Parallel and distributed computing for stochastic dual dynamic programming

Parallel machine scheduling with stochastic release times and processing times

Dynamic DAG Scheduling on Multiprocessor Systems: Reliability, Energy, and Makespan

Efficient Multi-Processor Scheduling in Increasingly Realistic Models

On Multiprocessor System Scheduling

Distributed Stochastic Optimization with Random Communication and Computational Delays: Optimal Policies and Performance Analysis

Machine Scheduling with Stochastic Release and Processing Times

Building Real-Time Parallel Task Systems on Multi-Cores: A Hierarchical Scheduling Approach.

Parallel Path Progression DAG Scheduling

Coded Computation across Shared Heterogeneous Workers with Communication Delay

Accelerating parallel graph computing with speculation

Decomposition-based scheduling for parallel real-time tasks on multiprocessors

Extending the Nested Parallel Model to the Nested Dataflow Model with Provably Efficient Schedulers

Improving Interference Analysis for Real-Time DAG Tasks Under Partitioned Scheduling

Superscalar Communication: A Runtime Optimization for Distributed Applications

An Algorithm Used to Improve Task Parallelization for Directed Acyclic Graphs