Abstract:Modern control solutions for HVAC have demonstrated excellent cost and energy savings through the utilisation of machine learning techniques. However, a challenging problem faced by most machine learning tasks is the amount of time and data required to train effective policies in the absence of prior knowledge. Considering that buildings from a specific geographical location share common environmental and structural features, this paper investigates the impact of spatial changes on performance accuracy through the use of transfer learning applied to reinforcement learning based HVAC control. We propose the development of an adapted RL (Q-learning) algorithm which can transfer HVAC control polices, adjusting themselves according to spatial changes. We examine the performance of our approach across multiple different locations. Moreover, an analysis of the user's time out comfort has been made, comparing models with and without transfer learning. The results from different cases show that after applying transfer learning the learning time to train optimal or near-optimal control policies was reduced by more than a factor of 6 when comparing to experiments without it. Furthermore, the test case where the spatial variation was lower than 50% achieved a similar performance for both dynamic and static HVAC control, presenting an average time out comfort error of 2.55% and 3.83%, respectively. From the user's perspective, it means they will not feel any additional discomfort, as the number of minutes out of the comfort zone for the static version is approximately the same for a 1-day interval. Finally, when examining the effect of transfer learning on geographical changes, the proposed method demonstrated higher performance in countries where the temperature variation is lower, reducing time out comfort by one-third. If an agent receives a policy from a place where the environmental conditions are very different the proposed method will still work and find the best policy, but not as fast as receiving it from a similar place.

Towards self-learning control of HVAC systems with the consideration of dynamic occupancy patterns: Application of model-free deep reinforcement learning

Modelling building HVAC control strategies using a deep reinforcement learning approach

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

Towards Optimal HVAC Control in Non-Stationary Building Environments Combining Active Change Detection and Deep Reinforcement Learning

Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning

Comparative study of model-based and model-free reinforcement learning control performance in HVAC systems

MF^2: Model-free Reinforcement Learning for Modeling-Free Building HVAC Control with Data-Driven Environment Construction in a Residential Building

Hybrid-Model-Based Deep Reinforcement Learning for Heating, Ventilation, and Air-Conditioning Control

Energy-Efficient HVAC Control Based on Reinforcement Learning and Transfer Learning in a Residential Building

Occupant-centric HVAC and window control: A reinforcement learning model for enhancing indoor thermal comfort and energy efficiency

ED-DQN: An event-driven deep reinforcement learning control method for multi-zone residential buildings

Energy-efficient control of thermal comfort in multi-zone residential HVAC via reinforcement learning

Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings

Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning

A Safe and Data-efficient Model-based Reinforcement Learning System for HVAC Control

MBRL-MC: An HVAC Control Approach via Combining Model-based Deep Reinforcement Learning and Model Predictive Control

DeepComfort: Energy-Efficient Thermal Comfort Control in Buildings Via Reinforcement Learning

USING DEEP LEARNING TO OPTIMIZE HVAC SYSTEMS IN RESIDENTIAL BUILDINGS

Model Predictive Control Strategy Using Encoder-Decoder Recurrent Neural Networks for Smart Control of Thermal Environment

Transfer Learning Applied to Reinforcement Learning-Based HVAC Control

Model predictive control with adaptive machine-learning-based model for building energy efficiency and comfort optimization