Abstract:Energy consumption for heating, ventilation and air conditioning (HVAC) has increased significantly and accounted for a large proportion of building energy growth. Advanced control strategies are needed to reduce energy consumption with maintaining occupant thermal comfort. While compared to other control problems, HVAC control is faced with numerous restrictions from real building environments. One key restriction is non-stationarity, i.e., the varying HVAC system dynamics. Researchers have paid efforts to solve the nonstationarity problems through different approaches, among which deep reinforcement learning gains traction for its advantages in capturing real-time information, controlling adaptively to system feedbacks, avoiding tedious modeling works and combining with deep learning techniques. However, current researches solved non-stationarity in a passive manner which hinders its potential and adds instability in real application. To fill this research gap, we propose a novel HVAC control method combining active building environment change detection and deep Q network (DQN), named non-stationary DQN. This method aims to disentangle the nonstationarity by actively identifying the change points of building environments and learning effective control strategies for corresponding building environments. The simulation results demonstrate that this developed non-stationary DQN method outperforms the state-of-art DQN method in both single-zone control and multi zone control tasks by saving unnecessary energy use and reducing thermal violation caused by non-stationarity. The improvement can reach 13% in energy-saving and 9% in thermal comfort. Besides, according to the results, our proposed method obtains stability against disturbance and generalization to an unseen building environment, which shows its robustness and potential in real-life applications.

Towards Optimal District Heating Temperature Control in China with Deep Reinforcement Learning

Deep Reinforcement Learning based Optimal Control of Hot Water Systems

Flow Rate Control in Smart District Heating Systems Using Deep Reinforcement Learning

Direct Thermal Load Control in Active Distribution Networks Based on Deep Reinforcement Learning

Deep Reinforcement Learning for Optimal Control of Space Heating

Heating ventilation air‐conditioner system for multi‐regional commercial buildings based on deep reinforcement learning

Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning

Intelligent control of district heating system based on RDPG

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

DeepComfort: Energy-Efficient Thermal Comfort Control in Buildings Via Reinforcement Learning

Energy-Efficient HVAC Control Based on Reinforcement Learning and Transfer Learning in a Residential Building

Distributed Multi-Agent Deep Reinforcement Learning Framework for Whole-building HVAC Control

Near-optimal Deep Reinforcement Learning Policies from Data for Zone Temperature Control

Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning

Transforming Cooling Optimization for Green Data Center via Deep Reinforcement Learning

Towards Optimal HVAC Control in Non-Stationary Building Environments Combining Active Change Detection and Deep Reinforcement Learning

Energy-efficient control of thermal comfort in multi-zone residential HVAC via reinforcement learning

Deep Reinforcement Learning for Heat Pump Control

Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings

Modelling building HVAC control strategies using a deep reinforcement learning approach

Reinforcement Learning for Optimal Control of a District Cooling Energy Plant