Abstract:Modern control solutions for HVAC have demonstrated excellent cost and energy savings through the utilisation of machine learning techniques. However, a challenging problem faced by most machine learning tasks is the amount of time and data required to train effective policies in the absence of prior knowledge. Considering that buildings from a specific geographical location share common environmental and structural features, this paper investigates the impact of spatial changes on performance accuracy through the use of transfer learning applied to reinforcement learning based HVAC control. We propose the development of an adapted RL (Q-learning) algorithm which can transfer HVAC control polices, adjusting themselves according to spatial changes. We examine the performance of our approach across multiple different locations. Moreover, an analysis of the user's time out comfort has been made, comparing models with and without transfer learning. The results from different cases show that after applying transfer learning the learning time to train optimal or near-optimal control policies was reduced by more than a factor of 6 when comparing to experiments without it. Furthermore, the test case where the spatial variation was lower than 50% achieved a similar performance for both dynamic and static HVAC control, presenting an average time out comfort error of 2.55% and 3.83%, respectively. From the user's perspective, it means they will not feel any additional discomfort, as the number of minutes out of the comfort zone for the static version is approximately the same for a 1-day interval. Finally, when examining the effect of transfer learning on geographical changes, the proposed method demonstrated higher performance in countries where the temperature variation is lower, reducing time out comfort by one-third. If an agent receives a policy from a place where the environmental conditions are very different the proposed method will still work and find the best policy, but not as fast as receiving it from a similar place.

Successful application of predictive information in deep reinforcement learning control: A case study based on an office building HVAC system

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

Modelling building HVAC control strategies using a deep reinforcement learning approach

Hybrid-Model-Based Deep Reinforcement Learning for Heating, Ventilation, and Air-Conditioning Control

Energy-Efficient HVAC Control Based on Reinforcement Learning and Transfer Learning in a Residential Building

Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning

Comparative study of model-based and model-free reinforcement learning control performance in HVAC systems

An experimental evaluation of deep reinforcement learning algorithms for HVAC control

Experimental evaluation of offline reinforcement learning for HVAC control in buildings

Deep Reinforcement Learning-Based Joint Optimization Control of Indoor Temperature and Relative Humidity in Office Buildings

Data-driven predictive control for smart HVAC system in IoT-integrated buildings with time-series forecasting and reinforcement learning

A Comparison of Classical and Deep Reinforcement Learning Methods for HVAC Control

MF^2: Model-free Reinforcement Learning for Modeling-Free Building HVAC Control with Data-Driven Environment Construction in a Residential Building

ED-DQN: An event-driven deep reinforcement learning control method for multi-zone residential buildings

USING DEEP LEARNING TO OPTIMIZE HVAC SYSTEMS IN RESIDENTIAL BUILDINGS

A Safe and Data-efficient Model-based Reinforcement Learning System for HVAC Control

Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning

Distributed Multi-Agent Deep Reinforcement Learning Framework for Whole-building HVAC Control

Heating ventilation air‐conditioner system for multi‐regional commercial buildings based on deep reinforcement learning

Transfer Learning Applied to Reinforcement Learning-Based HVAC Control