Abstract:Public cloud GPU clusters are becoming emerging platforms for training distributed deep learning jobs. Under this training paradigm, the job scheduler is a crucial component to improve user experiences, i.e., reducing training fees and job completion time, which can also save power costs for service providers. However, the scheduling problem is known to be NP-hard. Most existing work divides it into two easier sub-tasks, i.e., ordering task and placement task, which are responsible for deciding the scheduling orders of jobs and placement orders of GPU machines, respectively. Due to the superior adaptation ability, learning-based policies can generally perform better than traditional heuristic-based methods. Nevertheless, there are still two main challenges that have not been well-solved. First, most learning-based methods only focus on ordering or placement policy independently, while ignoring their cooperation. Second, the unbalanced machine performances and resource contention impose huge overhead and uncertainty on job duration, but rarely be considered in existing work. To tackle these issues, this paper presents a dual-agent scheduler framework abstracted from the two sub-tasks to jointly learn the ordering and placement policies and make better-informed scheduling decisions. Specifically, we design an ordering agent with a scalable squeeze-and-communicate strategy for better cooperation; for the placement agent, we propose a novel Random Walk Gaussian Process to learn the performance similarities of GPU machines while being aware of the uncertain performance fluctuation. Finally, the dual-agent is jointly optimized with multi-agent reinforcement learning. Extensive experiments conducted on the real-world production cluster trace demonstrate the superiority of our model.

TIAS: Two-level Information-Agnostic Job Scheduling in GPU Clusters

A DAG-Based XCIGS Algorithm for Dependent Tasks in Grid Environments.

Online Scheduling on a CPU-GPU Cluster

Adaptive Partitioning and Efficient Scheduling for Distributed DNN Training in Heterogeneous IoT Environment

Low-latency job scheduling with preemption for the development of deep learning

Scheduling Distributed Deep Learning Jobs in Heterogeneous Cluster with Placement Awareness

Scheduling Deep Learning Jobs in Multi-Tenant GPU Clusters via Wise Resource Sharing

UniSched: A Unified Scheduler for Deep Learning Training Jobs with Different User Demands

Astraea: A Fair Deep Learning Scheduler for Multi-Tenant GPU Clusters

Dynamic Resource Allocation for Deep Learning Clusters with Separated Compute and Storage

Interference-aware parallelization for deep learning workload in GPU cluster

An Adaptive Performance-oriented Scheduler for Static and Dynamic Heterogeneity

Online Job Scheduling in Distributed Machine Learning Clusters

SCHED²: Scheduling Deep Learning Training Via Deep Reinforcement Learning.

Task Scheduling for GPU Heterogeneous Cluster.

GPU Cluster Scheduling for Network-Sensitive Deep Learning

Optimization of Topology-Aware Job Allocation on a High-Performance Computing Cluster by Neural Simulated Annealing

PPS: Fair and efficient black-box scheduling for multi-tenant GPU clusters

AI-based Dynamic Schedule Calculation in Time Sensitive Networks using GCN-TD3

A Dual-Agent Scheduler for Distributed Deep Learning Jobs on Public Cloud Via Reinforcement Learning

Automated Runtime-Aware Scheduling for Multi-Tenant DNN Inference on GPU