Abstract:Automatic decision-making approaches, such as reinforcement learning (RL), have been applied to (partially) solve the resource allocation problem adaptively in the cloudcomputing system. However, a complete cloud resource allocation framework exhibits high dimensions in state and action spaces, which prohibit the usefulness of traditional RL techniques. In addition, high power consumption has become one of the critical concerns in design and control of cloud computing systems, which degrades system reliability and increases cooling cost. An effective dynamic power management (DPM) policy should minimize power consumption while maintaining performance degradationwithin an acceptable level. Thus, a joint virtual machine (VM) resource allocation and power management framework is critical to the overall cloud computing system. Moreover, novel solution framework is necessary to address the even higher dimensions in state and action spaces. In this paper, we propose a novel hierarchical framework forsolving the overall resource allocation and power management problem in cloud computing systems. The proposed hierarchical framework comprises a global tier for VM resource allocation to the servers and a local tier for distributed power management of local servers. The emerging deep reinforcement learning (DRL) technique, which can deal with complicated control problems with large state space, is adopted to solve the global tier problem. Furthermore, an autoencoder and a novel weight sharing structure are adopted to handle the high-dimensional state space and accelerate the convergence speed. On the other hand, the local tier of distributed server power managements comprises an LSTM based workload predictor and a model-free RL based power manager, operating in a distributed manner. Experiment results using actual Google cluster traces showthat our proposed hierarchical framework significantly savesthe power consumption and energy usage than the baselinewhile achieving no severe latency degradation. Meanwhile, the proposed framework can achieve the best trade-off between latency and power/energy consumption in a server cluster.

Multi-Dimensional Resource Allocation in Distributed Data Centers Using Deep Reinforcement Learning

Resource Management in Cloud Based on Deep Reinforcement Learning

Two-tiered Online Optimization of Region-wide Datacenter Resource Allocation via Deep Reinforcement Learning

Resource Allocation in Disaggregated Data Centre Systems with Reinforcement Learning

Smart Resource Allocation for Mobile Edge Computing: A Deep Reinforcement Learning Approach

Multi-user Resource Control with Deep Reinforcement Learning in IoT Edge Computing

Joint Task and Computing Resource Allocation in Distributed Edge Computing Systems Via Multi-Agent Deep Reinforcement Learning

Mobility-Aware Resource Allocation in Multi-Access Edge Computing Using Deep Reinforcement Learning

Reinforcement Learning-based Multidimensional Resource Management in Edge Cloud Regions.

Resource Management at the Network Edge: A Deep Reinforcement Learning Approach

A2C-DRL: Dynamic Scheduling for Stochastic Edge-Cloud Environments Using A2C and Deep Reinforcement Learning

EC-TRL: Evolutionary-Weighted Clustering and Transformer-Augmented Reinforcement Learning for Dynamic Resource Scheduling in Edge Cloud Environments

Dynamic Slicing of Multidimensional Resources in DCI-EON with Penalty-Aware Deep Reinforcement Learning

Distributed Edge Computing Resource Allocation Algorithm Based on DRL in LTE Access Network

Collaborative Multi-Agent Deep Reinforcement Learning for Energy-Efficient Resource Allocation in Heterogeneous Mobile Edge Computing Networks

"DRL + FL": an Intelligent Resource Allocation Model Based on Deep Reinforcement Learning for Mobile Edge Computing

Distributed and Collective Deep Reinforcement Learning for Computation Offloading: A Practical Perspective.

DeepMRA: an Efficient Microservices Resource Allocation Framework with Deep Reinforcement Learning in the Cloud

A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning

Deep Reinforcement Learning for Computation and Communication Resource Allocation in Multiaccess MEC Assisted Railway IoT Networks

Energy-Efficient Collaborative Multi-Access Edge Computing Via Deep Reinforcement Learning