Abstract:Reinforcement learning (RL) shows the potential to address drawbacks of rule-based control and model predictive control and exhibits great effectiveness in heating, ventilation and air conditioning (HVAC) systems. Most studies employed model-free RL to achieve building energy conservation and increase indoor comfort. However, model-free RL algorithms face the challenge of sample efficiency which causes long-time training and restricts their applications. Model-based RL is considered an alternative avenue for accelerating learning and promoting the application of RL, but it also has limitations due to modeling approaches and accuracy. In addition, few studies propose model-based RL algorithms and investigate performance gaps between model-free and model-based RL in HVAC systems. Therefore, this study conducts a comprehensive performance comparison between model-free and model-based RL to identify the current issues with RL control in HVAC systems. The open-source building optimization testing (BOPTEST) framework is employed as the virtual environment to evaluate the control performance and computational burden. Then Dueling Deep Q-Networks and Soft Actor-Critic are developed, and a state-of-the-art model-based RL framework is employed to develop their model-based versions. The comparison results showed that all RL controllers outperform the baseline control in terms of indoor temperature and operation costs. Model-based RL can achieve a control performance as good as model-free RL with a shorter training time based on its high sample efficiency. Moreover, due to massive and quickly generated data, model-based RL can accelerate the learning of RL agents, though the model is inaccurate at the early training stage. This study would provide some insights into the RL control selection and improvements in HVAC systems.

Energy Saving Algorithm of HVAC System Based on Deep Reinforcement Learning with Modelica Model

Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning

Hybrid-Model-Based Deep Reinforcement Learning for Heating, Ventilation, and Air-Conditioning Control

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

USING DEEP LEARNING TO OPTIMIZE HVAC SYSTEMS IN RESIDENTIAL BUILDINGS

Modelling building HVAC control strategies using a deep reinforcement learning approach

Deep Reinforcement Learning for Smart Home Energy Management

Heating ventilation air‐conditioner system for multi‐regional commercial buildings based on deep reinforcement learning

Distributed Multi-Agent Deep Reinforcement Learning Framework for Whole-building HVAC Control

ED-DQN: An event-driven deep reinforcement learning control method for multi-zone residential buildings

Energy-efficient control of thermal comfort in multi-zone residential HVAC via reinforcement learning

Energy-Efficient HVAC Control Based on Reinforcement Learning and Transfer Learning in a Residential Building

Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning

Performance optimization of HVAC systems with computational intelligence algorithms

A Community Energy Management Method Based on Explainable Reinforcement Learning

A Comparison of Classical and Deep Reinforcement Learning Methods for HVAC Control

An experimental evaluation of deep reinforcement learning algorithms for HVAC control

Towards self-learning control of HVAC systems with the consideration of dynamic occupancy patterns: Application of model-free deep reinforcement learning

Comparative study of model-based and model-free reinforcement learning control performance in HVAC systems

MBRL-MC: An HVAC Control Approach via Combining Model-based Deep Reinforcement Learning and Model Predictive Control

Deep Reinforcement Learning for Heat Pump Control