Dynamic Neighborhood Grouping-Based Multi-Objective Scheduling Algorithm for Workflow in Hybrid Cloud

Yulin Guo,Bo Liu,Weiwei Lin,Xiaoying Ye,James Z. Wang

DOI: https://doi.org/10.1016/j.future.2024.107633

IF: 7.307

2024-01-01

Future Generation Computer Systems

Abstract:The hybrid cloud is a crucial solution to overcome the limited resources of the private cloud and efficiently execute large-scale workflow due to its easy scalability and ability to guarantee data privacy. However, most of the existing studies on multi-objective workflow scheduling in a hybrid cloud treat the problem as a black box and perform overall optimization of large-scale decision variables, which will lead to the inefficient search of the solution space. In order to compensate for the above shortcomings, this paper proposes a workflow dynamic neighborhood search (WDNS) scheduling algorithm to simultaneously optimize the makespan, cost, and energy consumption of workflow execution in a hybrid cloud. Firstly, based on the data dependencies among workflow tasks, a dynamic neighborhood grouping strategy is proposed to divide decision variables with strong dependencies into the same group, thus effectively increasing the possibility of simultaneously optimizing interdependent variables. Then, based on the grouping strategy, new crossover and mutation operators are designed to search for feasible solutions, aiming to take advantage of divide-and-conquer to improve the search efficiency. Finally, in the context of 20 real-world workflows, our proposed WDNS algorithm was compared with four state-of-the-art algorithms. The comparison results confirm that WDNS outperformed the four algorithms across all 20 test cases in both hypervolume and inverted generational distance metrics.