Abstract:With the development of cloud computing, a growing number of applications are migrating to a cloud environment. In the process, the real-time scheduling of workflows has gradually become a technical challenge, due to the dynamic and uncertain nature of cloud environments and the complex dependencies between sub-tasks of the workflow. Although various methods have been reported up to now, these methods have their respective shortcomings, such as heuristic-based methods are hard to find optimal scheduling scheme and metaheuristic-based methods incur high computational overhead, which often lead to the violation of QoS (Quality of Service) requirements and increases service renting costs of executing workflows. Inspired by the successful application of Deep Reinforcement Learning (DRL) in cloud job scheduling, this paper proposes a real-time workflow scheduling method which combines Genetic Algorithm (GA) and DRL, aiming to reduce both execution cost and response time. To be specific, we design a real-time workflow scheduling algorithm named GA-DQN by combining the global search capability of GA and the environment awareness decision-making capability of DRL to divides scheduling process into two stages. First, the execution scheme of workflow in virtual machine is calculated when workflow arrives. Then, a DRL agent uses this scheme as the feature of workflow to assign workflow to a suitable virtual machine. In this way, the use of DRL to sense environment increases the computational efficiency of GA, and the execution scheme obtained by GA helps DRL to obtain the feature of workflow. On this basis of real world workflow, three groups of simulation experience are carried out to compare GA-DQN with four baseline method which consist of three traditional methods and a state-of-the-art method. The comparison results demonstrate that GA-DQN outperforms the other methods in terms of response time, execution cost, and success rate across different workloads and cloud instance configurations.

Cost-Aware Dynamic Multi-Workflow Scheduling in Cloud Data Center Using Evolutionary Reinforcement Learning

Cost-Aware Dynamic Cloud Workflow Scheduling using Self-Attention and Evolutionary Reinforcement Learning

Weighted Double Deep Q-network Based Reinforcement Learning for Bi-Objective Multi-Workflow Scheduling in the Cloud

Cost-aware scheduling systems for real-time workflows in cloud: An approach based on Genetic Algorithm and Deep Reinforcement Learning

Real-Time Multiple-Workflow Scheduling in Cloud Environments

Cooperative Coevolutionary Genetic Programming Hyper-Heuristic for Budget Constrained Dynamic Multi-workflow Scheduling in Cloud Computing

Reinforcement Learning Based Online Scheduling of Multiple Workflows in Edge Environment

A Deep Reinforcement Learning Approach for Cost Optimized Workflow Scheduling in Cloud Computing Environments

Learning to schedule multi-NUMA virtual machines via reinforcement learning

A2C-DRL: Dynamic Scheduling for Stochastic Edge-Cloud Environments Using A2C and Deep Reinforcement Learning

Multi-Agent Deep Reinforcement Learning Framework for Renewable Energy-Aware Workflow Scheduling on Distributed Cloud Data Centers

A DRL-based online VM scheduler for cost optimization in cloud brokers

Reinforcement Learning-driven Data-intensive Workflow Scheduling for Volunteer Edge-Cloud

Energy-aware systems for real-time job scheduling in cloud data centers: A deep reinforcement learning approach

An improved deep reinforcement learning-based scheduling approach for dynamic task scheduling in cloud manufacturing

Deep Reinforcement Learning for Multi-Resource Multi-Machine Job Scheduling

Reinforcement Learning based Workflow Scheduling in Cloud and Edge Computing Environments: A Taxonomy, Review and Future Directions

Dynamic multi-objective workflow scheduling for combined resources in cloud

A novel deep reinforcement learning scheme for task scheduling in cloud computing

A dynamic VM provisioning and de-provisioning based cost-efficient deadline-aware scheduling algorithm for Big Data workflow applications in a cloud environment

Decision variable contribution based adaptive mechanism for evolutionary multi-objective cloud workflow scheduling