Abstract:In the past few years, with the rapid development of CPU-GPU heterogeneous computing, the issue of task scheduling in the heterogeneous cluster has attracted a great deal of attention. This problem becomes more challenging with the need for efficient co-execution of tasks on the GPUs. However, the uncertainty of heterogeneous cluster and the interference caused by resource contention among co-executing tasks can lead to the unbalanced use of computing resource and further cause the degradation in performance of computing platform. In this article, we propose a two-stage task scheduling approach for streaming applications based on deep reinforcement learning and neural collaborative filtering, which considers fine-grained task division and task interference on the GPU. Specifically, the Learning-Driven Workload Parallelization (LDWP) method selects an appropriate execution node for the mutually independent tasks. By using the deep Q-network, the cluster-level scheduling model is online learned to perform the current optimal scheduling actions according to the runtime status of cluster environments and characteristics of tasks. The Interference-Aware Workload Parallelization (IAWP) method assigns subtasks with dependencies to the appropriate computing units, taking into account the interference of subtasks on the GPU by using neural collaborative filtering. For making the learning of neural network more efficient, we use pre-training in the two-stage scheduler. Besides, we use transfer learning technology to efficiently rebuild task scheduling model referring to the existing model. We evaluate our learning-driven and interference-aware task scheduling approach on a prototype platform with other widely used methods. The experimental results show that the proposed strategy can averagely improve the throughout for distributed computing system by 26.9 percent and improve the GPU resource utilization by around 14.7 percent.

Vapor: A GPU Sharing Scheduler with Communication and Computation Pipeline for Distributed Deep Learning

Liquid: Intelligent Resource Estimation and Network-Efficient Scheduling for Deep Learning Jobs on Distributed GPU Clusters

SchedTune: A Heterogeneity-Aware GPU Scheduler for Deep Learning

GPU Cluster Scheduling for Network-Sensitive Deep Learning

GPU Shared Scheduling System Under Deep Learning Container Cloud Platform

Scheduling Distributed Deep Learning Jobs in Heterogeneous Cluster with Placement Awareness

Learning-Driven Interference-Aware Workload Parallelization for Streaming Applications in Heterogeneous Cluster

A Dual-Agent Scheduler for Distributed Deep Learning Jobs on Public Cloud Via Reinforcement Learning

Adaptive Partitioning and Efficient Scheduling for Distributed DNN Training in Heterogeneous IoT Environment

Energy-Aware Non-Preemptive Task Scheduling with Deadline Constraint in DVFS-Enabled Heterogeneous Clusters

GAI: A Centralized Tree-Based Scheduler for Machine Learning Workload in Large Shared Clusters.

A Virtual Multi-Channel GPU Fair Scheduling Method for Virtual Machines.

Scheduling Deep Learning Jobs in Multi-Tenant GPU Clusters via Wise Resource Sharing

Optimizing execution for pipelined‐based distributed deep learning in a heterogeneously networked GPU cluster

HiveD: Sharing a GPU Cluster for Deep Learning with Guarantees

ElasticBatch: A Learning-Augmented Elastic Scheduling System for Batch Inference on MIG

On Scheduling Ring-All-Reduce Learning Jobs in Multi-Tenant GPU Clusters with Communication Contention

Interference-Aware Opportunistic Job Placement for Shared Distributed Deep Learning Clusters

Energy-Efficient GPU Clusters Scheduling for Deep Learning

Interference-aware parallelization for deep learning workload in GPU cluster

GreenFlow: A Carbon-Efficient Scheduler for Deep Learning Workloads