Employing Federated Learning for Training Autonomous HVAC Systems

Fredrik Hagström,Vikas Garg,Fabricio Oliveira

2024-05-01

Abstract:Buildings account for 40 % of global energy consumption. A considerable portion of building energy consumption stems from heating, ventilation, and air conditioning (HVAC), and thus implementing smart, energy-efficient HVAC systems has the potential to significantly impact the course of climate change. In recent years, model-free reinforcement learning algorithms have been increasingly assessed for this purpose due to their ability to learn and adapt purely from experience. They have been shown to outperform classical controllers in terms of energy cost and consumption, as well as thermal comfort. However, their weakness lies in their relatively poor data efficiency, requiring long periods of training to reach acceptable policies, making them inapplicable to real-world controllers directly. Hence, common research goals are to improve the learning speed, as well as to improve their ability to generalize, in order to facilitate transfer learning to unseen building environments. In this paper, we take a federated learning approach to training the reinforcement learning controller of an HVAC system. A global control policy is learned by aggregating local policies trained on multiple data centers located in different climate zones. The goal of the policy is to simultaneously minimize energy consumption and maximize thermal comfort. The federated optimization strategy indirectly increases both the rate at which experience data is collected and the variation in the data. We demonstrate through experimental evaluation that these effects lead to a faster learning speed, as well as greater generalization capabilities in the federated policy compared to any individually trained policy.

Optimization and Control,Machine Learning,Systems and Control

What problem does this paper attempt to address?

This paper aims to address a key issue in building energy consumption, especially the energy efficiency and intelligent control of heating, ventilation, and air conditioning (HVAC) systems. Traditional HVAC control systems are often inefficient and unable to adapt to environmental changes. In recent years, model-free reinforcement learning algorithms have received attention in HVAC control due to their ability to learn and adapt from experience, effectively reducing energy consumption and improving thermal comfort. However, these algorithms have low data efficiency and long training times, making them unsuitable for direct application in the real world.

The paper proposes a HVAC system controller training method based on federated learning. By training local policies on data centers in different climate regions and aggregating global control policies, it aims to simultaneously minimize energy consumption and maximize thermal comfort. The federated learning strategy improves data collection rate and data diversity, accelerating learning speed and enhancing generalization ability, while also improving learning stability and reducing sensitivity to parameter selection and reinforcement learning randomness.

Experimental results show that compared to individual policies, federated policies perform better in unseen environments, with faster learning speed, stronger generalization ability, and more stable learning process. Additionally, the study evaluates different local policy optimizers and federated learning algorithms to find the optimal configuration.

In conclusion, the paper attempts to address the issues of low data efficiency and poor generalization ability in reinforcement learning for HVAC control by utilizing federated learning to improve learning speed and adaptability of the controller, providing a more efficient and flexible solution for real-world HVAC system control.