Abstract:With the exploding of data-intensive web applications and requests (tasks), geo-distributed and large-scale data centers (DCs) are widely deployed in Software as a Service (SaaS) cloud, but server failures continue to grow at the same time. In this context, task scheduling problems become more intricate and both scheduling quality and scheduling speed raise further concerns. In this paper, we first propose a virtualized & monitoring SaaS model with predictive maintenance to minimize the costs of fault tolerance. Then with the monitored and predicted available states of servers, we focus on dynamic real-time task scheduling in geo-distributed and large-scale DCs with heterogeneous servers. Multiple objectives, including the long-term performance benefits, energy and communication costs, are taken into consideration in order to improve scheduling quality. For inter-DC and intra-DC task scheduling, two dynamic programming problems are formulated respectively, but there exists the problem that both state and action spaces are too large to be solved by simple iterations. To address this issue, we introduce the idea of reinforcement learning theory into solving traditional stochastic dynamic programming problems in the large-scale SaaS cloud, and put forward a cascaded two-level (inter-DC and intra-DC level) approximate dynamic programming (ADP) task-scheduling algorithm. The computation complexity can be significantly reduced and scheduling speed can be greatly improved. Finally, we conduct experiments with both random simulation data and Google cloud trace-logs. QoS evaluations and comparisons demonstrate that two ADP algorithms can work cooperatively, and our two-level ADP algorithm is more effective under large quantity of bursty requests.

A Distributed Task Scheduling System Suitable for Massive Environments

A Novel Task-Duplication based DAG Scheduling Algorithm for Heterogeneous Environments

Design And Implementation Of Task Scheduling Strategies For Massive Remote Sensing Data Processing Across Multiple Data Centers

Optimizing Schedule Length for DAG Type Applications with Energy Consumption Constraint in Heterogeneous Computing Systems

Modelling and Developing Conflict-Aware Scheduling on Large-Scale Data Centres

Hypergraph-partitioning-based online joint scheduling of tasks and data

Energy Efficient Speed Scaling and Task Scheduling for Distributed Computing Systems

Building Real-Time Parallel Task Systems on Multi-Cores: A Hierarchical Scheduling Approach.

Task Offloading in Real-Time Distributed Energy Power Systems

Two-level Task Scheduling with Multi-Objectives in Geo-Distributed and Large-Scale SaaS Cloud

A Novel Architecture Of Scheduling System For Big Data Framework

A model for intelligent task scheduling in a large distributed system

Joint Online Optimization of Task Rescheduling and Data Redistribution

A Large-Scale Task Scheduling Algorithm Based on Clustering and Duplication

A Data-Intensive Workflow Scheduling Algorithm for Large-Scale Cooperative Work Platform

Multi-objective Scheduling of Cloud Data Centers Prone to Failures

Dynamic Two-Side Matching of Tasks and Resources in Wide-Area Distributed Computing Environments.

Multisite Task Scheduling On Distributed Computing Grid

A Highly Cost-Effective Task Scheduling Strategy for Very Large Graph Computation.

Resource Load Based Stochastic DAGs Scheduling Mechanism for Grid Environment.

A Hybrid Genetic Scheduling Algorithm to Heterogeneous Distributed System