DeepComfort: Energy-Efficient Thermal Comfort Control in Buildings Via Reinforcement Learning

Guanyu Gao,Jie Li,Yonggang Wen

DOI: https://doi.org/10.1109/jiot.2020.2992117

IF: 10.6

2020-09-01

IEEE Internet of Things Journal

Abstract:Heating, ventilation, and air conditioning (HVAC) are extremely energy consuming, accounting for 40% of total building energy consumption. It is crucial to design some energy-efficient building thermal comfort control strategy which can reduce the energy consumption of the HVAC while maintaining the comfort of the occupants. However, implementing such a strategy is challenging, because the changes of the thermal states in a building environment are influenced by various factors. The relationships among these influencing factors are hard to model and are always different in different building environments. To address this challenge, we propose a deep-reinforcement-learning-based framework, DeepComfort, for thermal comfort control in buildings. We formulate the thermal comfort control as a cost-minimization problem by jointly considering the energy consumption of the HVAC and the occupants' thermal comfort. We first design a deep feedforward neural network (FNN)-based approach for predicting the occupants' thermal comfort and then propose a deep deterministic policy gradients (DDPGs)-based approach for learning the optimal thermal comfort control policy. We implement a building thermal comfort control simulation environment and evaluate the performance under various settings. The experimental results show that our approaches can improve the performance of thermal comfort prediction by 14.5% and reduce the energy consumption of HVAC by 4.31% while improving the occupants' thermal comfort by 13.6%.

computer science, information systems,telecommunications,engineering, electrical & electronic

What problem does this paper attempt to address?

The paper mainly addresses two major challenges faced by Heating, Ventilation, and Air Conditioning (HVAC) systems in building environments: 1. the issue of high energy consumption, and 2. how to reduce energy consumption while ensuring occupant comfort. Specifically, the goal of the paper is to design an intelligent control strategy that can effectively reduce the energy consumption of HVAC systems while maintaining indoor occupant comfort.

To achieve this goal, the authors propose a framework based on deep reinforcement learning—DeepComfort, for thermal comfort control in building environments. The framework mainly includes the following two aspects of work:

1. **Thermal Comfort Prediction**: Utilizing a Feedforward Neural Network (FNN) combined with Bayesian regularization to predict the thermal comfort of indoor occupants. This method can improve the accuracy of thermal comfort prediction, increasing it by 14.5% compared to existing methods.

2. **Thermal Comfort Control**: Employing the Deep Deterministic Policy Gradients (DDPG) algorithm to learn the optimal thermal comfort control strategy. DDPG is suitable for problems with continuous action spaces and can avoid discretizing the action space, thereby improving control accuracy and performance. Experimental results show that this method can reduce the energy consumption of HVAC systems by 4.31% while improving occupant thermal comfort by 13.6%.

In summary, this study proposes a comprehensive solution that combines thermal comfort prediction and deep reinforcement learning, aiming to achieve efficient operation of HVAC systems in building environments, i.e., reducing energy consumption while ensuring occupant thermal comfort.