Abstract:Deep learning (DL) has demonstrated significant success across diverse fields, leading to the construction of dedicated GPU accelerators within GPU clusters for high-quality training services. Efficient scheduler designs for such clusters are vital to reduce operational costs and enhance resource utilization. While recent schedulers have shown impressive performance in optimizing DL job performance and cluster utilization through periodic reallocation or selection of GPU resources, they also encounter challenges such as preemption and migration overhead, along with potential DL accuracy degradation. Nonetheless, few explore the potential benefits of GPU sharing to improve resource utilization and reduce job queuing times. Motivated by these insights, we present a job scheduling model allowing multiple jobs to share the same set of GPUs without altering job training settings. We introduce SJF-BSBF (shortest job first with best sharing benefit first), a straightforward yet effective heuristic scheduling algorithm. SJF-BSBF intelligently selects job pairs for GPU resource sharing and runtime settings (sub-batch size and scheduling time point) to optimize overall performance while ensuring DL convergence accuracy through gradient accumulation. In experiments with both physical DL workloads and trace-driven simulations, even as a preemption-free policy, SJF-BSBF reduces the average job completion time by 27-33\% relative to the state-of-the-art preemptive DL schedulers. Moreover, SJF-BSBF can wisely determine the optimal resource sharing settings, such as the sharing time point and sub-batch size for gradient accumulation, outperforming the aggressive GPU sharing approach (baseline SJF-FFS policy) by up to 17\% in large-scale traces.

JetEsti: A New DLT Job Scheduling Simulator Based on Fine-Grained Process Modeling.

Scheduling Distributed Deep Learning Jobs in Heterogeneous Cluster with Placement Awareness

UniSched: A Unified Scheduler for Deep Learning Training Jobs with Different User Demands

SCHED²: Scheduling Deep Learning Training Via Deep Reinforcement Learning.

Edge Computing Empowered Digital Twin: an End-to-End Computing Task Scheduling Approach

Online Job Scheduling in Distributed Machine Learning Clusters

Genetic Programming for Dynamic Workflow Scheduling in Fog Computing

Scheduling Deep Learning Jobs in Multi-Tenant GPU Clusters via Wise Resource Sharing

Learning Interpretable Scheduling Algorithms for Data Processing Clusters

An Improved Kubernetes Scheduling Algorithm for Deep Learning Platform

Low-latency job scheduling with preemption for the development of deep learning

Online Scheduling Algorithm for Heterogeneous Distributed Machine Learning Jobs

VELTAIR: towards high-performance multi-tenant deep learning services via adaptive compilation and scheduling

On a Meta Learning-based Scheduler for Deep Learning Clusters

Online Scheduling of Machine Learning Jobs in Edge-Cloud Networks

Dynamic Scheduling of MPI-based Distributed Deep Learning Training Jobs

DL2: A Deep Learning-driven Scheduler for Deep Learning Clusters

Job-aware Communication Scheduling for DML Training in Shared Cluster

Deep Reinforcement Learning-Based Workload Scheduling for Edge Computing

Evolutionary Trainer-Based Deep Q-Network for Dynamic Flexible Job Shop Scheduling

Learning to schedule (L2S): adaptive job shop scheduling using double deep Q network