Abstract:Deep learning is popular in many areas, but users must manually specify the resource configuration when submitting deep learning training jobs, usually over-provisioning resources. This kind of unreasonable resource configuration method results in slow training and low resource utilization. Therefore, it would be more convenient and efficient if users only need to specify the quality of service (QoS) for their jobs, and then the resources will be autoconfigured to meet the QoS. To satisfy this demand, we present a QoS-oriented scheduling and autoscaling framework that schedules and autoscales deep learning training jobs in the Kubernetes cluster. This paper focuses on the most important QoS requirement for deep learning training jobs: deadline.The goal of the framework is to guarantee that as many jobs as possible can be accomplished before their specified deadlines. To reach this goal, the framework schedules deep learning jobs by implementing a heuristic scheduling policy based on resource status and job deadline, and autoscales resource configuration by exploiting a characteristic of deep learning jobs: the predictability of training time. This predictability is used to predict whether a job can be accomplished before its deadline and estimate appropriate resource configuration if necessary.We implemented the framework by modifying the default scheduler of Kubernetes and conducted experiments to evaluate its performance. The experiment results show that our scheduling policy can improve the completion rate by 26% when the cluster resources are insufficient, and our autoscaling policy can improve the completion rate to 100% when the cluster resources are sufficient. We also show that the framework improves the utilization of allocated CPUs to 100%. Our proposed framework points to a new way of submitting and managing deep learning training jobs in the cluster.

DRS: A deep reinforcement learning enhanced Kubernetes scheduler for microservice‐based system

DL2: A Deep Learning-driven Scheduler for Deep Learning Clusters

DeepLRA: An Efficient Long Running Application Scheduling Framework with Deep Reinforcement Learning in the Cloud.

DRL-Scheduling: an Intelligent QoS-Aware Job Scheduling Framework for Applications in Clouds

DRPC: Distributed Reinforcement Learning Approach for Scalable Resource Provisioning in Container-based Clusters

A QoS-oriented Scheduling and Autoscaling Framework for Deep Learning

SCHED²: Scheduling Deep Learning Training Via Deep Reinforcement Learning.

An Integrated Dynamic Resource Scheduling Framework in On-Demand Clouds.

An Integrated Dynamic Resource Scheduling Framework in On-Demand Clouds

On a Meta Learning-based Scheduler for Deep Learning Clusters

A2C-DRL: Dynamic Scheduling for Stochastic Edge-Cloud Environments Using A2C and Deep Reinforcement Learning

An Improved Kubernetes Scheduling Algorithm for Deep Learning Platform

RLScheduler: An Automated HPC Batch Job Scheduler Using Reinforcement Learning

Dynamic scheduling for flexible job shop using a deep reinforcement learning approach

DeepMRA: an Efficient Microservices Resource Allocation Framework with Deep Reinforcement Learning in the Cloud

Poster Abstract: Deep Learning Workloads Scheduling with Reinforcement Learning on GPU Clusters

Optimization of Task-Scheduling Strategy in Edge Kubernetes Clusters Based on Deep Reinforcement Learning

Dynamic scheduling of decentralized high-end equipment R&D projects via deep reinforcement learning

RLConfig: Run-time Configuration of Cluster Schedulers via Deep Reinforcement Learning

DRAS-CQSim: A Reinforcement Learning based Framework for HPC Cluster Scheduling

Deep reinforcement learning for dynamic distributed job shop scheduling problem with transfers