Abstract:Cooling system plays a critical role in a modern data center (DC). Developing an optimal control policy for DC cooling system is a challenging task. The prevailing approaches often rely on approximating system models that are built upon the knowledge of mechanical cooling, electrical and thermal management, which is difficult to design and may lead to sub-optimal or unstable performances. In this paper, we propose utilizing the large amount of monitoring data in DC to optimize the control policy. To do so, we cast the cooling control policy design into an energy cost minimization problem with temperature constraints, and tap it into the emerging deep reinforcement learning (DRL) framework. Specifically, we propose an end-to-end cooling control algorithm (CCA) that is based on the actor-critic framework and an off-policy offline version of the deep deterministic policy gradient (DDPG) algorithm. In the proposed CCA, an evaluation network is trained to predict an energy cost counter penalized by the cooling status of the DC room, and a policy network is trained to predict optimized control settings when gave the current load and weather information. The proposed algorithm is evaluated on the EnergyPlus simulation platform and on a real data trace collected from the National Super Computing Centre (NSCC) of Singapore. Our results show that the proposed CCA can achieve about 11% cooling cost saving on the simulation platform compared with a manually configured baseline control algorithm. In the trace-based study, we propose a de-underestimation validation mechanism as we cannot directly test the algorithm on a real DC. Even though with DUE the results are conservative, we can still achieve about 15% cooling energy saving on the NSCC data trace if we set the inlet temperature threshold at 26.6 degree Celsius.

Frequency Regulation Capacity Offering of District Cooling System Based on Reinforcement Learning

Hierarchical Control Strategy for Load Regulation Based on Stackelberg Game Theory Considering Randomness

Hierarchical Control of Air-Conditioning Loads for Flexible Demand Response in the Short Term.

District Cooling System Control for Providing Operating Reserve Based on Safe Deep Reinforcement Learning

HVAC Scheduling for Demand Response in a Distributed CCHP System Considering Latent Load

Coordinated Control of Air-Conditioning Loads for System Frequency Regulation.

Reinforcement Learning-based District Cooling System Control for Voltage Regulation in Distribution Networks.

Distributed flexible resource regulation strategy for residential communities based on deep reinforcement learning

Learning-based strategic operation of district cooling systems with ice storage systems

Research on Frequency Regulation Market Application Strategy of Integrated Fire-Storage System Based on Q -Reinforcement Learning

Evaluation of Frequency Regulation Provision by Commercial Building HVAC Systems

Frequency Provision Through Two-Timescale Commercial Building HVAC System Control Using Hierarchical Reinforcement Learning

Increase the integration of renewable energy using flexibility of source-network-load-storage in district cooling system

Direct Thermal Load Control in Active Distribution Networks Based on Deep Reinforcement Learning

Optimal Control of District Cooling Energy Plant with Reinforcement Learning and MPC

Deep Reinforcement Learning-Based Scheduling Optimization of a Wind-solar Coupled CHP Unit System Considering Source-load Uncertainty

Reinforcement Learning for Optimal Control of a District Cooling Energy Plant

Reinforcement learning approach for optimal control of ice-based thermal energy storage (TES) systems in commercial buildings

Optimal Operation for District Cooling Systems Coupled with Ice Storage Units Based on the Per-Unit Value Form

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment

Transforming Cooling Optimization for Green Data Center via Deep Reinforcement Learning