Abstract:Modern control solutions for HVAC have demonstrated excellent cost and energy savings through the utilisation of machine learning techniques. However, a challenging problem faced by most machine learning tasks is the amount of time and data required to train effective policies in the absence of prior knowledge. Considering that buildings from a specific geographical location share common environmental and structural features, this paper investigates the impact of spatial changes on performance accuracy through the use of transfer learning applied to reinforcement learning based HVAC control. We propose the development of an adapted RL (Q-learning) algorithm which can transfer HVAC control polices, adjusting themselves according to spatial changes. We examine the performance of our approach across multiple different locations. Moreover, an analysis of the user's time out comfort has been made, comparing models with and without transfer learning. The results from different cases show that after applying transfer learning the learning time to train optimal or near-optimal control policies was reduced by more than a factor of 6 when comparing to experiments without it. Furthermore, the test case where the spatial variation was lower than 50% achieved a similar performance for both dynamic and static HVAC control, presenting an average time out comfort error of 2.55% and 3.83%, respectively. From the user's perspective, it means they will not feel any additional discomfort, as the number of minutes out of the comfort zone for the static version is approximately the same for a 1-day interval. Finally, when examining the effect of transfer learning on geographical changes, the proposed method demonstrated higher performance in countries where the temperature variation is lower, reducing time out comfort by one-third. If an agent receives a policy from a place where the environmental conditions are very different the proposed method will still work and find the best policy, but not as fast as receiving it from a similar place.

Enhanced Strategies for Off-Policy Reinforcement Learning Algorithms in HVAC Control

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

Modelling building HVAC control strategies using a deep reinforcement learning approach

Transfer Learning Applied to Reinforcement Learning-Based HVAC Control

A Comparison of Classical and Deep Reinforcement Learning Methods for HVAC Control

Hybrid-Model-Based Deep Reinforcement Learning for Heating, Ventilation, and Air-Conditioning Control

Go Beyond Black-box Policies: Rethinking the Design of Learning Agent for Interpretable and Verifiable HVAC Control

Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning

Energy Saving Algorithm of HVAC System Based on Deep Reinforcement Learning with Modelica Model

Comparative study of model-based and model-free reinforcement learning control performance in HVAC systems

Experimental evaluation of offline reinforcement learning for HVAC control in buildings

Enhancing the Performance of Multi-Agent Reinforcement Learning for Controlling HVAC Systems

Optimizing Industrial HVAC Systems with Hierarchical Reinforcement Learning

Towards Optimal HVAC Control in Non-Stationary Building Environments Combining Active Change Detection and Deep Reinforcement Learning

ED-DQN: An event-driven deep reinforcement learning control method for multi-zone residential buildings

A Relearning Approach to Reinforcement Learning for Control of Smart Buildings

Optimal Control of Multiroom HVAC System: an Event-Based Approach

Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning

Occupant-centric HVAC and window control: A reinforcement learning model for enhancing indoor thermal comfort and energy efficiency

Successful application of predictive information in deep reinforcement learning control: A case study based on an office building HVAC system

Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings