Energy Optimization for HVAC Systems in Multi-VAV Open Offices: A Deep Reinforcement Learning Approach

Hao Wang,Xiwen Chen,Natan Vital,Edward.Duffy,Abolfazl Razi

2023-11-15

Abstract:With more than 32% of the global energy used by commercial and residential buildings, there is an urgent need to revisit traditional approaches to Building Energy Management (BEM). With HVAC systems accounting for about 40% of the total energy cost in the commercial sector, we propose a low-complexity DRL-based model with multi-input multi-output architecture for the HVAC energy optimization of open-plan offices, which uses only a handful of controllable and accessible factors. The efficacy of our solution is evaluated through extensive analysis of the overall energy consumption and thermal comfort levels compared to a baseline system based on the existing HVAC schedule in a real building. This comparison shows that our method achieves 37% savings in energy consumption with minimum violation (<1%) of the desired temperature range during work hours. It takes only a total of 40 minutes for 5 epochs (about 7.75 minutes per epoch) to train a network with superior performance and covering diverse conditions for its low-complexity architecture; therefore, it easily adapts to changes in the building setups, weather conditions, occupancy rate, etc. Moreover, by enforcing smoothness on the control strategy, we suppress the frequent and unpleasant on/off transitions on HVAC units to avoid occupant discomfort and potential damage to the system. The generalizability of our model is verified by applying it to different building models and under various weather conditions.

Systems and Control,Artificial Intelligence

What problem does this paper attempt to address?

This paper attempts to address the issue of optimizing the energy consumption of HVAC systems in multi-zone variable air volume (VAV) open office environments using deep reinforcement learning (DRL) methods. Specifically, the paper aims to achieve the following objectives: 1. **Modeling and Analysis**: Establish and analyze the thermodynamic characteristics in multi-zone open office environments. 2. **Control Algorithm**: Propose a DRL-based control algorithm to optimize both thermal comfort and energy efficiency. 3. **Energy Efficiency Improvement**: Achieve a 37% reduction in HVAC system energy consumption with a temperature comfort violation rate of less than 1% through the proposed algorithm. 4. **Smooth Control**: Introduce heuristic reward strategies and smooth control algorithms to reduce frequent switching of HVAC systems, avoiding discomfort for users and potential system damage. 5. **Generalization Capability**: Validate the applicability of the method under different floor layouts and various weather conditions. The main contributions of the paper include: - Analyzing the thermal transfer characteristics of connected spaces in open office environments and comparing them with traditional closed office environments. - Proposing a multi-input multi-output architecture DRL control algorithm that achieves significant energy efficiency improvements. - Requiring only a few input variables, including outdoor temperature, indoor temperature, time, and control signals, making the framework simple and easy to generalize to other buildings. - Introducing heuristic reward strategies to accelerate the training process and reduce model complexity. - Adding penalty terms in the cost function to reduce frequent switching operations, avoiding user discomfort and system damage. - High computational efficiency of the model, with each training round taking approximately 7.75 minutes, and a total of 5 training rounds taking only about 40 minutes, making it easy to adapt to other open office environments. Overall, this paper achieves significant energy efficiency improvements in multi-zone variable air volume open office environments through DRL methods while ensuring user thermal comfort.

Energy Optimization for HVAC Systems in Multi-VAV Open Offices: A Deep Reinforcement Learning Approach

Energy optimization for HVAC systems in multi-VAV open offices: A deep reinforcement learning approach

Heating ventilation air‐conditioner system for multi‐regional commercial buildings based on deep reinforcement learning

Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning

Energy-Efficient HVAC Control Based on Reinforcement Learning and Transfer Learning in a Residential Building

Optimal HVAC Control in Shared Office Spaces Based on Deep Reinforcement Learning

USING DEEP LEARNING TO OPTIMIZE HVAC SYSTEMS IN RESIDENTIAL BUILDINGS

Distributed Multi-Agent Deep Reinforcement Learning Framework for Whole-building HVAC Control

Exploring Deep Reinforcement Learning for Holistic Smart Building Control

Energy-efficient Personalized Thermal Comfort Control in Office Buildings Based on Multi-Agent Deep Reinforcement Learning

Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings

Hybrid-Model-Based Deep Reinforcement Learning for Heating, Ventilation, and Air-Conditioning Control

Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning

Study on the Application of Reinforcement Learning in the Operation Optimization of HVAC System

Real-World Data and Calibrated Simulation Suite for Offline Training of Reinforcement Learning Agents to Optimize Energy and Emission in Buildings for Environmental Sustainability

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

ED-DQN: An event-driven deep reinforcement learning control method for multi-zone residential buildings

Modelling building HVAC control strategies using a deep reinforcement learning approach

Occupant-centric HVAC and window control: A reinforcement learning model for enhancing indoor thermal comfort and energy efficiency

Modeling and Optimizing Building HVAC Energy Systems Using Deep Neural Networks

One for Many: Transfer Learning for Building HVAC Control