Abstract:To conserve building energy, optimal operation of a building’s energy systems, especially heating, ventilation and air-conditioning (HVAC) systems, is important. This study focuses on the optimization of the central chiller plant, which accounts for a large portion of the HVAC system’s energy consumption. Classic optimal control methods for central chiller plants are mostly based on system performance models which takes much effort and cost to establish. In addition, inevitable model error could cause control risk to the applied system. To mitigate the model dependency of HVAC optimal control, reinforcement learning (RL) algorithms have been drawing attention in the HVAC control domain due to its model-free feature. Currently, the RL-based optimization of central chiller plants faces several challenges: (1) existing model-free control methods based on RL typically adopt single-agent scheme, which brings high training cost and long training period when optimizing multiple controllable variables for large-scaled systems; (2) multi-agent scheme could overcome the former problem, but it also requires a proper coordination mechanism to harmonize the potential conflicts among all involved RL agents; (3) previous agent coordination frameworks (identified by distributed control or decentralized control) are mainly designed for model-based control methods instead of model-free controllers. To tackle the problems above, this article proposes a multi-agent, model-free optimal control approach for central chiller plants. This approach utilizes game theory and the RL algorithm SARSA for agent coordination and learning, respectively. A data-driven system model is set up using measured field data of a real HVAC system for simulation. The simulation case study results suggest that the energy saving performance (both short- and long-term) of the proposed approach (over 10% in a cooling season compared to the rule-based baseline controller) is close to the classic multi-agent reinforcement learning (MARL) algorithm WoLF-PHC; moreover, the proposed approach’s nature of few pending parameters makes it more feasible and robust for engineering practices than the WoLF-PHC algorithm.

Reinforcement learning approach for optimal control of ice-based thermal energy storage (TES) systems in commercial buildings

A novel approach of day-ahead cooling load prediction and optimal control for ice-based thermal energy storage (TES) system in commercial buildings

Reinforcement Learning for Optimal Control of a District Cooling Energy Plant

Cooling Water System Control of a Commercial Complex Based on Reinforcement Learning

Coordinated energy management for integrated energy system incorporating multiple flexibility measures of supply and demand sides: A deep reinforcement learning approach

Data Analytics and Optimization of an Ice-Based Energy Storage System for Commercial Buildings

Predictive control optimization of chiller plants based on deep reinforcement learning

Optimization of operational strategy for ice thermal energy storage in a district cooling system based on model predictive control

Deep learning and physics-based modeling for the optimization of ice-based thermal energy systems in cooling plants

Multi-Agent Optimal Control for Central Chiller Plants Using Reinforcement Learning and Game Theory

Deep Reinforcement Learning for Autonomous Water Heater Control

Reinforcement learning of room temperature set-point of thermal storage air-conditioning system with demand response

Optimal Control of District Cooling Energy Plant with Reinforcement Learning and MPC

Energy Scheduling Strategy of Ice Storage Air Conditioning System Based on Deep Reinforcement Learning.

Reinforcement learning-based demand response strategy for thermal energy storage air-conditioning system considering room temperature and humidity setpoints

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

Hybrid-Model-Based Deep Reinforcement Learning for Heating, Ventilation, and Air-Conditioning Control

Study on the Application of Reinforcement Learning in the Operation Optimization of HVAC System

Direct Thermal Load Control in Active Distribution Networks Based on Deep Reinforcement Learning

Local Integrated Energy System Operational Optimization Considering Multi-Type Uncertainties: A Reinforcement Learning Approach Based on Improved TD3 Algorithm

Learning a Distributed Control Scheme for Demand Flexibility in Thermostatically Controlled Loads