Abstract:Energy consumption for heating, ventilation and air conditioning (HVAC) has increased significantly and accounted for a large proportion of building energy growth. Advanced control strategies are needed to reduce energy consumption with maintaining occupant thermal comfort. While compared to other control problems, HVAC control is faced with numerous restrictions from real building environments. One key restriction is non-stationarity, i.e., the varying HVAC system dynamics. Researchers have paid efforts to solve the nonstationarity problems through different approaches, among which deep reinforcement learning gains traction for its advantages in capturing real-time information, controlling adaptively to system feedbacks, avoiding tedious modeling works and combining with deep learning techniques. However, current researches solved non-stationarity in a passive manner which hinders its potential and adds instability in real application. To fill this research gap, we propose a novel HVAC control method combining active building environment change detection and deep Q network (DQN), named non-stationary DQN. This method aims to disentangle the nonstationarity by actively identifying the change points of building environments and learning effective control strategies for corresponding building environments. The simulation results demonstrate that this developed non-stationary DQN method outperforms the state-of-art DQN method in both single-zone control and multi zone control tasks by saving unnecessary energy use and reducing thermal violation caused by non-stationarity. The improvement can reach 13% in energy-saving and 9% in thermal comfort. Besides, according to the results, our proposed method obtains stability against disturbance and generalization to an unseen building environment, which shows its robustness and potential in real-life applications.

A multi-grid reinforcement learning method for energy conservation and comfort of HVAC in buildings

Grid-Interactive Multi-Zone Building Control Using Reinforcement Learning with Global-Local Policy Search

Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings

Energy-Efficient HVAC Control Based on Reinforcement Learning and Transfer Learning in a Residential Building

Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning

Multi-agent Deep Reinforcement Learning-Based Coordination Control for Grid-Aware Multi-Buildings

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning

Heating ventilation air‐conditioner system for multi‐regional commercial buildings based on deep reinforcement learning

Occupant-centric HVAC and window control: A reinforcement learning model for enhancing indoor thermal comfort and energy efficiency

Towards Optimal HVAC Control in Non-Stationary Building Environments Combining Active Change Detection and Deep Reinforcement Learning

Optimal control method of HVAC based on multi-agent deep reinforcement learning

Energy-efficient control of thermal comfort in multi-zone residential HVAC via reinforcement learning

Hybrid-Model-Based Deep Reinforcement Learning for Heating, Ventilation, and Air-Conditioning Control

A Multi-Agent Deep Constrained Q-Learning Method for Smart Building Energy Management Under Uncertainties

Comparative study of model-based and model-free reinforcement learning control performance in HVAC systems

Two-Stage Reinforcement Learning Policy Search for Grid-Interactive Building Control

Experimental evaluation of offline reinforcement learning for HVAC control in buildings

Optimal HVAC Control in Shared Office Spaces Based on Deep Reinforcement Learning

A Safe and Data-efficient Model-based Reinforcement Learning System for HVAC Control

ED-DQN: An event-driven deep reinforcement learning control method for multi-zone residential buildings