Abstract:Energy consumption for heating, ventilation and air conditioning (HVAC) has increased significantly and accounted for a large proportion of building energy growth. Advanced control strategies are needed to reduce energy consumption with maintaining occupant thermal comfort. While compared to other control problems, HVAC control is faced with numerous restrictions from real building environments. One key restriction is non-stationarity, i.e., the varying HVAC system dynamics. Researchers have paid efforts to solve the nonstationarity problems through different approaches, among which deep reinforcement learning gains traction for its advantages in capturing real-time information, controlling adaptively to system feedbacks, avoiding tedious modeling works and combining with deep learning techniques. However, current researches solved non-stationarity in a passive manner which hinders its potential and adds instability in real application. To fill this research gap, we propose a novel HVAC control method combining active building environment change detection and deep Q network (DQN), named non-stationary DQN. This method aims to disentangle the nonstationarity by actively identifying the change points of building environments and learning effective control strategies for corresponding building environments. The simulation results demonstrate that this developed non-stationary DQN method outperforms the state-of-art DQN method in both single-zone control and multi zone control tasks by saving unnecessary energy use and reducing thermal violation caused by non-stationarity. The improvement can reach 13% in energy-saving and 9% in thermal comfort. Besides, according to the results, our proposed method obtains stability against disturbance and generalization to an unseen building environment, which shows its robustness and potential in real-life applications.

Optimal HVAC Control in Shared Office Spaces Based on Deep Reinforcement Learning

Energy-efficient Personalized Thermal Comfort Control in Office Buildings Based on Multi-Agent Deep Reinforcement Learning

Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings

Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning

Energy Optimization for HVAC Systems in Multi-VAV Open Offices: A Deep Reinforcement Learning Approach

Towards Optimal HVAC Control in Non-Stationary Building Environments Combining Active Change Detection and Deep Reinforcement Learning

Deep Reinforcement Learning-Based Joint Optimization Control of Indoor Temperature and Relative Humidity in Office Buildings

DeepComfort: Energy-Efficient Thermal Comfort Control in Buildings Via Reinforcement Learning

Heating ventilation air‐conditioner system for multi‐regional commercial buildings based on deep reinforcement learning

Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning

Occupant-centric HVAC and window control: A reinforcement learning model for enhancing indoor thermal comfort and energy efficiency

Deep reinforcement learning optimal control strategy for temperature setpoint real-time reset in multi-zone building HVAC system

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

USING DEEP LEARNING TO OPTIMIZE HVAC SYSTEMS IN RESIDENTIAL BUILDINGS

Heating Control Strategy Based on Dynamic Programming for Building Energy Saving and Emission Reduction

Optimal Control of Multiroom HVAC System: an Event-Based Approach

A Unified Control Framework of HVAC System for Thermal and Acoustic Comforts in Office Building.

Double Deep Q-learning Based on Personalized Thermal Comfort Model for HVAC Optimization

Distributed Multi-Agent Deep Reinforcement Learning Framework for Whole-building HVAC Control

ED-DQN: An event-driven deep reinforcement learning control method for multi-zone residential buildings