Abstract:In the past few years, with the rapid development of CPU-GPU heterogeneous computing, the issue of task scheduling in the heterogeneous cluster has attracted a great deal of attention. This problem becomes more challenging with the need for efficient co-execution of tasks on the GPUs. However, the uncertainty of heterogeneous cluster and the interference caused by resource contention among co-executing tasks can lead to the unbalanced use of computing resource and further cause the degradation in performance of computing platform. In this article, we propose a two-stage task scheduling approach for streaming applications based on deep reinforcement learning and neural collaborative filtering, which considers fine-grained task division and task interference on the GPU. Specifically, the Learning-Driven Workload Parallelization (LDWP) method selects an appropriate execution node for the mutually independent tasks. By using the deep Q-network, the cluster-level scheduling model is online learned to perform the current optimal scheduling actions according to the runtime status of cluster environments and characteristics of tasks. The Interference-Aware Workload Parallelization (IAWP) method assigns subtasks with dependencies to the appropriate computing units, taking into account the interference of subtasks on the GPU by using neural collaborative filtering. For making the learning of neural network more efficient, we use pre-training in the two-stage scheduler. Besides, we use transfer learning technology to efficiently rebuild task scheduling model referring to the existing model. We evaluate our learning-driven and interference-aware task scheduling approach on a prototype platform with other widely used methods. The experimental results show that the proposed strategy can averagely improve the throughout for distributed computing system by 26.9 percent and improve the GPU resource utilization by around 14.7 percent.

Fast Distributed Deep Learning Via Worker-adaptive Batch Sizing

WBSP: Addressing Stragglers in Distributed Machine Learning with Worker-Busy Synchronous Parallel

Semi-Dynamic Load Balancing: Efficient Distributed Learning in Non-Dedicated Environments

LBB: load-balanced batching for efficient distributed learning on heterogeneous GPU cluster

DBS: Dynamic Batch Size For Distributed Deep Neural Network Training

DLB: A Dynamic Load Balance Strategy for Distributed Training of Deep Neural Networks

Locality-aware and Fault-tolerant Batching for Machine Learning on Distributed Datasets

EP4DDL: addressing straggler problem in heterogeneous distributed deep learning

Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methods

Adaptive Partitioning and Efficient Scheduling for Distributed DNN Training in Heterogeneous IoT Environment

On Distributed Deep Network for Processing Large-Scale Sets of Complex Data

Round-Robin Synchronization: Mitigating Communication Bottlenecks in Parameter Servers.

Online Job Scheduling in Distributed Machine Learning Clusters

Joint Dynamic Data and Model Parallelism for Distributed Training of DNNs over Heterogeneous Infrastructure

Straggler-Aware Gradient Aggregation for Large-Scale Distributed Deep Learning System

Accelerating Massively Distributed Deep Learning Through Efficient Pseudo-Synchronous Update Method

Scheduling Distributed Deep Learning Jobs in Heterogeneous Cluster with Placement Awareness

Learning-Driven Interference-Aware Workload Parallelization for Streaming Applications in Heterogeneous Cluster

Fast and Straggler-Tolerant Distributed SGD with Reduced Computation Load

ElasticBatch: A Learning-Augmented Elastic Scheduling System for Batch Inference on MIG

Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of Resources