Abstract:Energy consumption for heating, ventilation and air conditioning (HVAC) has increased significantly and accounted for a large proportion of building energy growth. Advanced control strategies are needed to reduce energy consumption with maintaining occupant thermal comfort. While compared to other control problems, HVAC control is faced with numerous restrictions from real building environments. One key restriction is non-stationarity, i.e., the varying HVAC system dynamics. Researchers have paid efforts to solve the nonstationarity problems through different approaches, among which deep reinforcement learning gains traction for its advantages in capturing real-time information, controlling adaptively to system feedbacks, avoiding tedious modeling works and combining with deep learning techniques. However, current researches solved non-stationarity in a passive manner which hinders its potential and adds instability in real application. To fill this research gap, we propose a novel HVAC control method combining active building environment change detection and deep Q network (DQN), named non-stationary DQN. This method aims to disentangle the nonstationarity by actively identifying the change points of building environments and learning effective control strategies for corresponding building environments. The simulation results demonstrate that this developed non-stationary DQN method outperforms the state-of-art DQN method in both single-zone control and multi zone control tasks by saving unnecessary energy use and reducing thermal violation caused by non-stationarity. The improvement can reach 13% in energy-saving and 9% in thermal comfort. Besides, according to the results, our proposed method obtains stability against disturbance and generalization to an unseen building environment, which shows its robustness and potential in real-life applications.

Indoor temperature preference setting control method for thermal comfort and energy saving based on reinforcement learning

Energy-efficient control of thermal comfort in multi-zone residential HVAC via reinforcement learning

INDOOR THERMAL ENVIRONMENT CONTROL BASE ON THERMAL SENSATION PREDICTED BY ARTIFICIAL INTELLIGENCE

Deep reinforcement learning optimal control strategy for temperature setpoint real-time reset in multi-zone building HVAC system

Reinforcement learning-based demand response strategy for thermal energy storage air-conditioning system considering room temperature and humidity setpoints

Reinforcement learning of room temperature set-point of thermal storage air-conditioning system with demand response

Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning

Study on Indoor Thermal Environment Control Based on Thermal Sensation Prediction

Hybrid-Model-Based Deep Reinforcement Learning for Heating, Ventilation, and Air-Conditioning Control

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

Occupant-centric HVAC and window control: A reinforcement learning model for enhancing indoor thermal comfort and energy efficiency

Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning

Optimal HVAC Control in Shared Office Spaces Based on Deep Reinforcement Learning

Smart Control of Indoor Thermal Environment Based on Online Learned Thermal Comfort Model Using Infrared Thermal Imaging

Energy-Efficient HVAC Control Based on Reinforcement Learning and Transfer Learning in a Residential Building

Intelligent control of electric vehicle air conditioning system based on deep reinforcement learning

Heating Control Strategy Based on Dynamic Programming for Building Energy Saving and Emission Reduction

Experimental evaluation of offline reinforcement learning for HVAC control in buildings

Operation Strategy of Smart Thermostats That Self-Learn User Preferences.

Experiments of indoor thermal environment control during cooling operation based on thermal sensation and temperature set point

Towards Optimal HVAC Control in Non-Stationary Building Environments Combining Active Change Detection and Deep Reinforcement Learning