Abstract:Efficient GPU scheduling is the key to minimizing the execution time of the Deep Learning (DL) training workloads. DL training system schedulers typically allocate a fixed number of GPUs to each job, which inhibits high resource utilization and often extends the overall training time. The recent introduction of schedulers that can dynamically reallocate GPUs has achieved better cluster efficiency. This dynamic nature, however, introduces additional overhead by terminating and restarting jobs or requires modification to the DL training frameworks.We propose and develop an efficient, non-intrusive GPU scheduling framework that employs a combination of an adaptive GPU scheduler and an elastic GPU allocation mechanism to reduce the completion time of DL training workloads and improve resource utilization. Specifically, the adaptive GPU scheduler includes a scheduling algorithm that uses training job progress information to determine the most efficient allocation and reallocation of GPUs for incoming and running jobs at any given time. The elastic GPU allocation mechanism works in concert with the scheduler. It offers a lightweight and nonintrusive method to reallocate GPUs based on a “SideCar” process that temporarily stops and restarts the job’s DL training process with a different number of GPUs. We implemented the scheduling framework as plugins in Kubernetes and conducted evaluations on two 16-GPU clusters with multiple training jobs based on TensorFlow. Results show that our proposed scheduling framework reduces the overall execution time and the average job completion time by up to 45% and 63%, respectively, compared to the Kubernetes default scheduler. Compared to a termination based scheduler, our framework reduces the overall execution time and the average job completion time by up to 20% and 37%, respectively.

GPU Cluster Scheduling for Network-Sensitive Deep Learning

Scheduling Distributed Deep Learning Jobs in Heterogeneous Cluster with Placement Awareness

An Efficient and Non-Intrusive GPU Scheduling Framework for Deep Learning Training Systems.

Energy-Efficient GPU Clusters Scheduling for Deep Learning

SchedTune: A Heterogeneity-Aware GPU Scheduler for Deep Learning

Liquid: Intelligent Resource Estimation and Network-Efficient Scheduling for Deep Learning Jobs on Distributed GPU Clusters

Deep Learning Workload Scheduling in GPU Datacenters: Taxonomy, Challenges and Vision

Communication Contention Aware Scheduling of Multiple Deep Learning Training Jobs

Poster Abstract: Deep Learning Workloads Scheduling with Reinforcement Learning on GPU Clusters

SCHED²: Scheduling Deep Learning Training Via Deep Reinforcement Learning.

Deep Learning Workload Scheduling in GPU Datacenters: A Survey

An Optimal Network-Aware Scheduling Technique for Distributed Deep Learning in Distributed HPC Platforms

UniSched: A Unified Scheduler for Deep Learning Training Jobs with Different User Demands

Deep Learning Research and Development Platform: Characterizing and Scheduling with QoS Guarantees on GPU Clusters.

Effective Scheduler for Distributed DNN Training Based on MapReduce and GPU Cluster

E-LAS: Design and Analysis of Completion-Time Agnostic Scheduling for Distributed Deep Learning Cluster

Intra-Cluster Coalescing and Distributed-Block Scheduling to Reduce GPU NoC Pressure.

Isolated Scheduling for Distributed Training Tasks in GPU Clusters

Energy-Aware Non-Preemptive Task Scheduling with Deadline Constraint in DVFS-Enabled Heterogeneous Clusters

GAI: A Centralized Tree-Based Scheduler for Machine Learning Workload in Large Shared Clusters.

Scheduling Deep Learning Jobs in Multi-Tenant GPU Clusters via Wise Resource Sharing