Abstract:Cooling system plays a critical role in a modern data center (DC). Developing an optimal control policy for DC cooling system is a challenging task. The prevailing approaches often rely on approximating system models that are built upon the knowledge of mechanical cooling, electrical and thermal management, which is difficult to design and may lead to sub-optimal or unstable performances. In this paper, we propose utilizing the large amount of monitoring data in DC to optimize the control policy. To do so, we cast the cooling control policy design into an energy cost minimization problem with temperature constraints, and tap it into the emerging deep reinforcement learning (DRL) framework. Specifically, we propose an end-to-end cooling control algorithm (CCA) that is based on the actor-critic framework and an off-policy offline version of the deep deterministic policy gradient (DDPG) algorithm. In the proposed CCA, an evaluation network is trained to predict an energy cost counter penalized by the cooling status of the DC room, and a policy network is trained to predict optimized control settings when gave the current load and weather information. The proposed algorithm is evaluated on the EnergyPlus simulation platform and on a real data trace collected from the National Super Computing Centre (NSCC) of Singapore. Our results show that the proposed CCA can achieve about 11% cooling cost saving on the simulation platform compared with a manually configured baseline control algorithm. In the trace-based study, we propose a de-underestimation validation mechanism as we cannot directly test the algorithm on a real DC. Even though with DUE the results are conservative, we can still achieve about 15% cooling energy saving on the NSCC data trace if we set the inlet temperature threshold at 26.6 degree Celsius.

Research on Overall Energy Consumption Optimization Method for Data Center Based on Deep Reinforcement Learning

Optimizing Data Center Energy Efficiency Via Event-Driven Deep Reinforcement Learning

Deep Reinforcement Learning Based Joint Optimization for Energy-efficient ITE and Cooling Units in Data Centers

Comparison of Deep Reinforcement Learning Algorithms in Data Center Cooling Management: A Case Study

Cooperatively Improving Data Center Energy Efficiency Based on Multi-Agent Deep Reinforcement Learning

Optimizing Energy Efficiency for Data Center via Parameterized Deep Reinforcement Learning

DeepEE: Joint Optimization of Job Scheduling and Cooling Control for Data Center Energy Efficiency Using Deep Reinforcement Learning

Jointly Optimizing the IT and Cooling Systems for Data Center Energy Efficiency Based on Multi-Agent Deep Reinforcement Learning

Joint IT-Facility Optimization for Green Data Centers via Deep Reinforcement Learning

Resource Management in Cloud Based on Deep Reinforcement Learning

SafeCool: Safe and Energy-Efficient Cooling Management in Data Centers with Model-Based Reinforcement Learning

Energy saving evaluation of an energy efficient data center using a model-free reinforcement learning approach

Global Optimization Strategy of Prosumer Data Center System Operation Based on Multi-Agent Deep Reinforcement Learning

Transforming Cooling Optimization for Green Data Center via Deep Reinforcement Learning

A Reinforcement Learning Based Framework for Holistic Energy Optimization of Sustainable Cloud Data Centers

An Energy Saving Control Strategy Based on Multi-Agent Q-Learning Algorithm for Data Center

DeepCCHP: Intelligent Comprehensive Optimization of Energy Consumption and Carbon Emission for Data Center CCHP Systems

Towards an energy-efficient Data Center Network based on deep reinforcement learning

Energy-efficient VM Scheduling Based on Deep Reinforcement Learning

The Optimized Reinforcement Learning Approach to Run-Time Scheduling in Data Center.

A multi-agent reinforcement learning-based method for server energy efficiency optimization combining DVFS and dynamic fan control