Abstract:With global warming intensifying and resource conflicts escalating, the world is undergoing a transformative shift toward sustainable practices and energy-efficient solutions. With more than 32% of the global energy used by commercial and residential buildings, there is an urgent need to revisit traditional approaches to Building Energy Management (BEM). Within a BEMSplatform, regulating the operation of Heating, Ventilation, and Air Conditioning (HVAC) systems is more important, noting that HVAC systems account for about 40% of the total energy cost in the commercial sector. This paper offers a Deep Reinforcement Learning (DRL) algorithm as a data-driven approach to controlling HVAC operation to enhance the energy efficiency of commercial buildings with open offices while ensuring thermal comfort for occupants in different zones. Compared to alternative methods such as rule-based models and model-predictive control, data-driven models have shown promising results in optimizing building energy consumption without the need for building-specific thresholds, prior knowledge about the underlying physics of heat distribution, and digital mapping of the airflow. Despite the astonishing performance of modern DRL methods in controlling energy management, a particular energy-saving solution for open-plan offices with multiple Variable Air Volume (VAV) systems, where different zones cannot be treated independently, is still missing. Also, some of the existing methods suffer from key issues such as long training time and lack of generalizability for using over-complicated models, incorporating external factors that are hard to model and characterize, and including factors that are not typically accessible. To solve these issues, we propose a low-complexity DRL-based model with multi-input multi-output architecture for the HVAC energy optimization of open-plan offices, which uses only a handful of controllable and accessible factors. The efficacy of our solution is evaluated through extensive analysis of the overall energy consumption and thermal comfort levels compared to a baseline system based on the existing HVAC schedule from a real case. This comparison shows that our method achieves 37% savings in energy consumption with minimum temperature violation ( < 1%) of the desired temperature range during work hours. It takes only a total of 40 min for 5 epochs (about 7.75 min per epoch) to train a network with superior performance and covering diverse conditions for its low-complexity architecture; therefore, it easily adapts to changes in the building setups, weather conditions, occupancy rate, etc. Moreover, by enforcing smoothness on the control strategy, we suppress the frequent and unpleasant on/off transitions on HVAC units to avoid occupant discomfort and potential damage to the system. The generalizability of our model is verified by applying it to different building models and under various weather conditions.

Towards Optimal HVAC Control in Non-Stationary Building Environments Combining Active Change Detection and Deep Reinforcement Learning

ED-DQN: An event-driven deep reinforcement learning control method for multi-zone residential buildings

Deep reinforcement learning optimal control strategy for temperature setpoint real-time reset in multi-zone building HVAC system

Energy-efficient control of thermal comfort in multi-zone residential HVAC via reinforcement learning

Toward Smart Multizone HVAC Control by Combining Context-Aware System and Deep Reinforcement Learning.

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

Optimal HVAC Control in Shared Office Spaces Based on Deep Reinforcement Learning

Energy-Efficient HVAC Control Based on Reinforcement Learning and Transfer Learning in a Residential Building

Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings

Occupant-centric HVAC and window control: A reinforcement learning model for enhancing indoor thermal comfort and energy efficiency

Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning

A Comparative Study of DQN and D3QN for HVAC System Optimization Control

Towards self-learning control of HVAC systems with the consideration of dynamic occupancy patterns: Application of model-free deep reinforcement learning

Modelling building HVAC control strategies using a deep reinforcement learning approach

Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning

MF^2: Model-free Reinforcement Learning for Modeling-Free Building HVAC Control with Data-Driven Environment Construction in a Residential Building

DeepComfort: Energy-Efficient Thermal Comfort Control in Buildings Via Reinforcement Learning

Heating Control Strategy Based on Dynamic Programming for Building Energy Saving and Emission Reduction

One for Many: Transfer Learning for Building HVAC Control

Heating ventilation air‐conditioner system for multi‐regional commercial buildings based on deep reinforcement learning

Energy optimization for HVAC systems in multi-VAV open offices: A deep reinforcement learning approach