Abstract:Reinforcement learning (RL) shows the potential to address drawbacks of rule-based control and model predictive control and exhibits great effectiveness in heating, ventilation and air conditioning (HVAC) systems. Most studies employed model-free RL to achieve building energy conservation and increase indoor comfort. However, model-free RL algorithms face the challenge of sample efficiency which causes long-time training and restricts their applications. Model-based RL is considered an alternative avenue for accelerating learning and promoting the application of RL, but it also has limitations due to modeling approaches and accuracy. In addition, few studies propose model-based RL algorithms and investigate performance gaps between model-free and model-based RL in HVAC systems. Therefore, this study conducts a comprehensive performance comparison between model-free and model-based RL to identify the current issues with RL control in HVAC systems. The open-source building optimization testing (BOPTEST) framework is employed as the virtual environment to evaluate the control performance and computational burden. Then Dueling Deep Q-Networks and Soft Actor-Critic are developed, and a state-of-the-art model-based RL framework is employed to develop their model-based versions. The comparison results showed that all RL controllers outperform the baseline control in terms of indoor temperature and operation costs. Model-based RL can achieve a control performance as good as model-free RL with a shorter training time based on its high sample efficiency. Moreover, due to massive and quickly generated data, model-based RL can accelerate the learning of RL agents, though the model is inaccurate at the early training stage. This study would provide some insights into the RL control selection and improvements in HVAC systems.

A review of reinforcement learning methodologies for controlling occupant comfort in buildings

A Novel Reinforcement Learning Method for Improving Occupant Comfort Via Window Opening and Closing

A review of reinforcement learning for controlling Building Energy Systems from a computer science perspective

Applications of reinforcement learning for building energy efficiency control: A review

Challenges and opportunities of occupant-centric building controls in real-world implementation: A critical review

Simulation Studies on Deep Reinforcement Learning for Building Control with Human Interaction

Occupant-centric HVAC and window control: A reinforcement learning model for enhancing indoor thermal comfort and energy efficiency

Experimental evaluation of offline reinforcement learning for HVAC control in buildings

Reinforcement learning building control approach harnessing imitation learning

A review of current research on occupant-centric control for improving comfort and energy efficiency

Evaluating Reinforcement Learning Algorithms in Residential Energy Saving and Comfort Management

Satisfaction Based Q-learning for Integrated Lighting and Blind Control

Reinforcement learning-based thermal comfort control for vehicle cabins

Exploring Deep Reinforcement Learning for Holistic Smart Building Control

A Unified Control Framework of HVAC System for Thermal and Acoustic Comforts in Office Building.

Deep Reinforcement Learning-Based Joint Optimization Control of Indoor Temperature and Relative Humidity in Office Buildings

Comparative study of model-based and model-free reinforcement learning control performance in HVAC systems

An experimental evaluation of deep reinforcement learning algorithms for HVAC control

Enhancing the Performance of Multi-Agent Reinforcement Learning for Controlling HVAC Systems

DeepComfort: Energy-Efficient Thermal Comfort Control in Buildings Via Reinforcement Learning

The use of artificial intelligence (AI) methods in the prediction of thermal comfort in buildings: energy implications of AI-based thermal comfort controls