Abstract:Energy consumption for heating, ventilation and air conditioning (HVAC) has increased significantly and accounted for a large proportion of building energy growth. Advanced control strategies are needed to reduce energy consumption with maintaining occupant thermal comfort. While compared to other control problems, HVAC control is faced with numerous restrictions from real building environments. One key restriction is non-stationarity, i.e., the varying HVAC system dynamics. Researchers have paid efforts to solve the nonstationarity problems through different approaches, among which deep reinforcement learning gains traction for its advantages in capturing real-time information, controlling adaptively to system feedbacks, avoiding tedious modeling works and combining with deep learning techniques. However, current researches solved non-stationarity in a passive manner which hinders its potential and adds instability in real application. To fill this research gap, we propose a novel HVAC control method combining active building environment change detection and deep Q network (DQN), named non-stationary DQN. This method aims to disentangle the nonstationarity by actively identifying the change points of building environments and learning effective control strategies for corresponding building environments. The simulation results demonstrate that this developed non-stationary DQN method outperforms the state-of-art DQN method in both single-zone control and multi zone control tasks by saving unnecessary energy use and reducing thermal violation caused by non-stationarity. The improvement can reach 13% in energy-saving and 9% in thermal comfort. Besides, according to the results, our proposed method obtains stability against disturbance and generalization to an unseen building environment, which shows its robustness and potential in real-life applications.

A Unified Control Framework of HVAC System for Thermal and Acoustic Comforts in Office Building.

Optimal HVAC Control in Shared Office Spaces Based on Deep Reinforcement Learning

Double-Layered Model Predictive Control for Building HVAC Systems Considering Thermal Comfort*

A multi-occupants’ comfort-driven and energy-efficient control strategy of VAV system based on learned thermal comfort profiles

Energy-efficient Personalized Thermal Comfort Control in Office Buildings Based on Multi-Agent Deep Reinforcement Learning

User-centric Approach to Optimizing Thermal Comfort in University Classrooms: Utilizing Computer Vision and Q-XGBoost Reinforcement Learning

DeepComfort: Energy-Efficient Thermal Comfort Control in Buildings Via Reinforcement Learning

Deep Reinforcement Learning-Based Joint Optimization Control of Indoor Temperature and Relative Humidity in Office Buildings

Multi-sensor Information Fusion Based Control for VAV Systems Using Thermal Comfort Constraints

Computer vision-based smart HVAC control system for university classroom in a subtropical climate

An Interactive Task Conditioning System Featuring Personal Comfort Models and Non-Intrusive Sensing Techniques: A Field Study in Shanghai

Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning

Cyber-Enabled Optimization of HVAC System Control in Open Space of Office Building

Building Temperature and Humidity Adaptive Control for a Multi-Zone HVAC System Using Hybrid Modeling Method

A Systematic Approach for Exploring Tradeoffs in Predictive HVAC Control Systems for Buildings

Integrated control of blind and lights in daily office environment

Thermal, Lighting and IAQ Control System for Energy Saving and Comfort Management

Towards Optimal HVAC Control in Non-Stationary Building Environments Combining Active Change Detection and Deep Reinforcement Learning

Energy-efficient control of thermal comfort in multi-zone residential HVAC via reinforcement learning

Learning-based Framework for Sensor Fault-Tolerant Building HVAC Control with Model-assisted Learning

Harmonizing Comfort and Energy: A Multi-Objective Framework for Central Air Conditioning Systems