Abstract:To conserve building energy, optimal operation of a building’s energy systems, especially heating, ventilation and air-conditioning (HVAC) systems, is important. This study focuses on the optimization of the central chiller plant, which accounts for a large portion of the HVAC system’s energy consumption. Classic optimal control methods for central chiller plants are mostly based on system performance models which takes much effort and cost to establish. In addition, inevitable model error could cause control risk to the applied system. To mitigate the model dependency of HVAC optimal control, reinforcement learning (RL) algorithms have been drawing attention in the HVAC control domain due to its model-free feature. Currently, the RL-based optimization of central chiller plants faces several challenges: (1) existing model-free control methods based on RL typically adopt single-agent scheme, which brings high training cost and long training period when optimizing multiple controllable variables for large-scaled systems; (2) multi-agent scheme could overcome the former problem, but it also requires a proper coordination mechanism to harmonize the potential conflicts among all involved RL agents; (3) previous agent coordination frameworks (identified by distributed control or decentralized control) are mainly designed for model-based control methods instead of model-free controllers. To tackle the problems above, this article proposes a multi-agent, model-free optimal control approach for central chiller plants. This approach utilizes game theory and the RL algorithm SARSA for agent coordination and learning, respectively. A data-driven system model is set up using measured field data of a real HVAC system for simulation. The simulation case study results suggest that the energy saving performance (both short- and long-term) of the proposed approach (over 10% in a cooling season compared to the rule-based baseline controller) is close to the classic multi-agent reinforcement learning (MARL) algorithm WoLF-PHC; moreover, the proposed approach’s nature of few pending parameters makes it more feasible and robust for engineering practices than the WoLF-PHC algorithm.

Large-Scale Data Center Cooling Control Via Sample-Efficient Reinforcement Learning

Transforming Cooling Optimization for Green Data Center via Deep Reinforcement Learning

Integrating Mechanism and Data: Reinforcement Learning Based on Multi-Fidelity Model for Data Center Cooling Control

Development of a Soft Actor Critic Deep Reinforcement Learning Approach for Harnessing Energy Flexibility in a Large Office Building

Cooling Water System Control of a Commercial Complex Based on Reinforcement Learning

Sustainability of Data Center Digital Twins with Reinforcement Learning

Controlling Commercial Cooling Systems Using Reinforcement Learning

Reinforcement Learning for Optimal Control of a District Cooling Energy Plant

Optimizing Data Centre Energy Efficiency via Event-Driven Deep Reinforcement Learning

Multi-Agent Optimal Control for Central Chiller Plants Using Reinforcement Learning and Game Theory

Reinforcement Learning Control Strategy for Differential Pressure Setpoint in Large-scale Multi-source Looped District Cooling System

Deep Reinforcement Learning for Tropical Air Free-cooled Data Center Control

An Alternative Reinforcement Learning (ARL) Control Strategy for Data Center Air-Cooled HVAC Systems

Deep Reinforcement Learning Based Joint Optimization for Energy-efficient ITE and Cooling Units in Data Centers

Optimizing Energy Efficiency for Data Center via Parameterized Deep Reinforcement Learning

A multi-setpoint cooling control approach for air-cooled data centers using the deep Q-network algorithm

Residual Physics and Post-Posed Shielding for Safe Deep Reinforcement Learning Method

Optimal control method of HVAC based on multi-agent deep reinforcement learning

Research on Overall Energy Consumption Optimization Method for Data Center Based on Deep Reinforcement Learning

Hybrid-Model-Based Deep Reinforcement Learning for Heating, Ventilation, and Air-Conditioning Control

Energy-Efficient HVAC Control Based on Reinforcement Learning and Transfer Learning in a Residential Building