Abstract:In recent years, the number of clusters and cloud platforms dedicated to deep learning acceleration has increased, and research on multi-tenant deep learning (DL) cluster scheduling systems has also advanced quickly. However, we have observed several shortcomings in these systems. Firstly, resources exhibit heterogeneity, but even the most advanced heterogeneity-aware schedulers can only reach the GPU-type level. In addition, most scheduling systems cannot perform well in balancing efficiency and fairness, which leads to unfair resource allocation and reduced user satisfaction. Moreover, we have noticed the phenomenon of cluster fragmentation and job starvation. In this paper, we propose a new scheduling architecture: Hops, which includes (1) fine-grained heterogeneity awareness and accurate throughput estimators, which allows for heterogeneity awareness at the server entity level. (2) Hops performs resource allocation by executing prior weighted integer linear programming (ILP) for specific placement locations, effectively balancing fairness and efficiency. (3) Hops introduces "latency ratio fairness" (LRF) as a user fairness criterion, which helps reduce starvation and enhance user experience. (4) To address cluster fragmentation, Hops intentionally uses low-sensitivity jobs to fill fragments. The final experimental results show that, in physical experiments, compared with the state-of-the-art scheduling architectures: Sia [17] and Gavel [32], Hops reduces cluster completion time by 18.5% to 34.2%, shortens average job completion time (JCT) by 27.4% to 45.9%, lowers waiting latency by 35.4% to 54.9%, significantly reduces cluster fragmentation, and performs significantly better in fairness metrics compared to Sia and Gavel. In the 512-GPU simulation experiments, Hops not only improves system efficiency but also reduces the maximum job latency ratio by over 21× and decreases cluster fragmentation to less than 1 GPU per round on average.

Design and realization of hybrid resource management system for heterogeneous cluster

Task Scheduling for GPU Heterogeneous Cluster.

Resource Scheduling Strategy for Performance Optimization Based on Heterogeneous CPU-GPU Platform

An Energy Efficient Task Scheduling Scheme for Heterogeneous GPU-enhanced Clusters

Dynamic Resource Management in a HPC and Cloud Hybrid Environment

A user mode CPU–GPU scheduling framework for hybrid workloads

Task Scheduling Greedy Heuristics for GPU Heterogeneous Cluster Involving the Weights of the Processor

An Improved LBM Approach for Heterogeneous GPU-CPU Clusters

Hybrid Map Task Scheduling for GPU-Based Heterogeneous Clusters

QoS-aware Dynamic Resource Allocation with Improved Utilization and Energy Efficiency on GPU

An Energy-Efficient Task Scheduling Method for CPU-GPU Heterogeneous Cloud.

Exploration on Task Scheduling Strategy for CPU-GPU Heterogeneous Computing System

AN PARALLEL AND DISTRIBUTED PROGRAMMING SOLUTION BASED ON HETEROGENEOUS GPU CLUSTER

Design of load balancing algorithm for CPU and GPU collaborative computing based on C4.5 decision tree.

Learning Driven Parallelization for Large-Scale Video Workload in Hybrid CPU-GPU Cluster

Energy Efficient Real-Time Task Scheduling on CPU-GPU Hybrid Clusters

MixHeter: A Global Scheduler for Mixed Workloads in Heterogeneous Environments

Programming Framework for Node Heterogeneous GPU Cluster

Hops: Fine-grained Heterogeneous Sensing, Efficient and Fair Deep Learning Cluster Scheduling System

Learning-Driven Interference-Aware Workload Parallelization for Streaming Applications in Heterogeneous Cluster

Multiple Resources Scheduling for Diverse Workloads in Heterogeneous Datacenter