A heuristic multi-objective task scheduling framework for container-based clouds via actor-critic reinforcement learning
Lilu Zhu,Feng Wu,Yanfeng Hu,Kai Huang,Xinmei Tian
DOI: https://doi.org/10.1007/s00521-023-08208-6
2023-03-18
Neural Computing and Applications
Abstract:Container-based cloud technology has changed the delivery mode of traditional applications and brought a breakthrough development to the field of cloud computing. However, the uncertainty of cloud environment and variability of application requirements increase the scheduling cost of tasks in container cloud. In particular, how to balance the business performance and utilization efficiency of cloud resources in the peak stage of application access is the focus of future for container cluster technology. In this paper, we propose a heuristics multi-objective task scheduling framework based on reinforcement learning (AC-CCTS). The proposed framework not only solves the problems of single objective and local convergence in traditional task scheduling methods, but also reduces the cost of experiential learning with reinforcement learning methods. Firstly, we define container cloud environment, scheduling agent, scheduling actions and scheduling evaluation methods to establish a deep reinforcement learning-based dynamic scheduling model. Then, based on Actor-Critic algorithm, we design heuristic rules and prioritized experience replay method to speed up convergence of task scheduling and decrease learning costs. At the same time, we provide compensation mechanism for dynamic task scheduling to improve the robustness of the approach. Finally, we implement comparative experiments to simulate various scheduling scenarios and verify the effectiveness of AC-CCTS from different perspectives such as resource balance, resource utilization and QoS. Compared with traditional meta-heuristic scheduling methods such as FIMPSO, HWOA-MBA and other reinforcement learning algorithms such as DeepRM-Plus and RLSched, AC-CCTS shows better resource utilization efficiency and convergence stability in container-based cloud task scheduling.
computer science, artificial intelligence