Abstract:Automatic decision-making approaches, such as reinforcement learning (RL), have been applied to (partially) solve the resource allocation problem adaptively in the cloudcomputing system. However, a complete cloud resource allocation framework exhibits high dimensions in state and action spaces, which prohibit the usefulness of traditional RL techniques. In addition, high power consumption has become one of the critical concerns in design and control of cloud computing systems, which degrades system reliability and increases cooling cost. An effective dynamic power management (DPM) policy should minimize power consumption while maintaining performance degradationwithin an acceptable level. Thus, a joint virtual machine (VM) resource allocation and power management framework is critical to the overall cloud computing system. Moreover, novel solution framework is necessary to address the even higher dimensions in state and action spaces. In this paper, we propose a novel hierarchical framework forsolving the overall resource allocation and power management problem in cloud computing systems. The proposed hierarchical framework comprises a global tier for VM resource allocation to the servers and a local tier for distributed power management of local servers. The emerging deep reinforcement learning (DRL) technique, which can deal with complicated control problems with large state space, is adopted to solve the global tier problem. Furthermore, an autoencoder and a novel weight sharing structure are adopted to handle the high-dimensional state space and accelerate the convergence speed. On the other hand, the local tier of distributed server power managements comprises an LSTM based workload predictor and a model-free RL based power manager, operating in a distributed manner. Experiment results using actual Google cluster traces showthat our proposed hierarchical framework significantly savesthe power consumption and energy usage than the baselinewhile achieving no severe latency degradation. Meanwhile, the proposed framework can achieve the best trade-off between latency and power/energy consumption in a server cluster.

Energy-aware systems for real-time job scheduling in cloud data centers: A deep reinforcement learning approach

Adaptive DRL-Based Task Scheduling for Energy-Efficient Cloud Computing

A Deep Reinforcement Learning-Based Preemptive Approach for Cost-Aware Cloud Job Scheduling

Resource Management in Cloud Based on Deep Reinforcement Learning

Energy-Efficient Joint Task Assignment and Migration in Data Centers: A Deep Reinforcement Learning Approach

Energy efficient task scheduling based on deep reinforcement learning in cloud environment: A specialized review

Energy-aware Task Scheduling Optimization with Deep Reinforcement Learning for Large-Scale Heterogeneous Systems

DRL-Scheduling: an Intelligent QoS-Aware Job Scheduling Framework for Applications in Clouds

Deep Reinforcement Learning for Job Scheduling and Resource Management in Cloud Computing: An Algorithm-Level Review

RARE: Renewable Energy Aware Resource Management in Datacenters

A novel deep reinforcement learning scheme for task scheduling in cloud computing

DeepEE: Joint Optimization of Job Scheduling and Cooling Control for Data Center Energy Efficiency Using Deep Reinforcement Learning

Real-Time Tasks Oriented Energy-Aware Scheduling in Virtualized Clouds

The Optimized Reinforcement Learning Approach to Run-Time Scheduling in Data Center.

A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning

Energy-Aware Rolling-Horizon Scheduling for Real-Time Tasks in Virtualized Cloud Data Centers

Cost-aware scheduling systems for real-time workflows in cloud: An approach based on Genetic Algorithm and Deep Reinforcement Learning

A Reinforcement Learning-Based Mixed Job Scheduler Scheme for Grid or IaaS Cloud

Reinforcement Learning Based Task Scheduling for Environmentally Sustainable Federated Cloud Computing

Deep Reinforcement Learning-based Methods for Resource Scheduling in Cloud Computing: A Review and Future Directions

A Reinforcement Learning Based Framework for Holistic Energy Optimization of Sustainable Cloud Data Centers