Abstract:Energy consumption for heating, ventilation and air conditioning (HVAC) has increased significantly and accounted for a large proportion of building energy growth. Advanced control strategies are needed to reduce energy consumption with maintaining occupant thermal comfort. While compared to other control problems, HVAC control is faced with numerous restrictions from real building environments. One key restriction is non-stationarity, i.e., the varying HVAC system dynamics. Researchers have paid efforts to solve the nonstationarity problems through different approaches, among which deep reinforcement learning gains traction for its advantages in capturing real-time information, controlling adaptively to system feedbacks, avoiding tedious modeling works and combining with deep learning techniques. However, current researches solved non-stationarity in a passive manner which hinders its potential and adds instability in real application. To fill this research gap, we propose a novel HVAC control method combining active building environment change detection and deep Q network (DQN), named non-stationary DQN. This method aims to disentangle the nonstationarity by actively identifying the change points of building environments and learning effective control strategies for corresponding building environments. The simulation results demonstrate that this developed non-stationary DQN method outperforms the state-of-art DQN method in both single-zone control and multi zone control tasks by saving unnecessary energy use and reducing thermal violation caused by non-stationarity. The improvement can reach 13% in energy-saving and 9% in thermal comfort. Besides, according to the results, our proposed method obtains stability against disturbance and generalization to an unseen building environment, which shows its robustness and potential in real-life applications.

Energy Efficient Thermal Management of 5G Base Station Site Based on Reinforcement Learning

Coordinated Optimization for Energy Efficient Thermal Management of 5G Base Station Site

An Efficient and Flexible Learning Framework for Dynamic Power and Thermal Co-Management

Energy-Efficient Placement Optimization of the HVAC System for 5G Base Station with Redundant Configuration

Efficient Initial Access with Deep Reinforcement Learning Based Beam Sweeping in Wireless Cellular Communication Systems

A Deep Reinforcement Learning-Based Power Control Scheme for the 5G Wireless Systems

Deep Reinforcement Learning With Spatio-Temporal Traffic Forecasting for Data-Driven Base Station Sleep Control

Online Learning for Intelligent Thermal Management of Interference-coupled and Passively Cooled Base Stations

BESS Aided Reconfigurable Energy Supply using Deep Reinforcement Learning for 5G and Beyond

GCD Optimization and Intelligent Management for Green Base Station with Battery Control

Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning

Energy-Efficient HVAC Control Based on Reinforcement Learning and Transfer Learning in a Residential Building

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

A Coordinated Energy Management Method for 5G Base Station Using Multi-Agent Deep Deterministic Policy Gradient*

Towards Optimal HVAC Control in Non-Stationary Building Environments Combining Active Change Detection and Deep Reinforcement Learning

Using Reinforcement Learning to Reduce Energy Consumption of Ultra-Dense Networks With 5G Use Cases Requirements

An efficient energy saving scheme using reinforcement learning for 5G and beyond in H-CRAN

Reinforcement Learning-Based Interference Control for Ultra-Dense Small Cells.

Joint Traffic Prediction and Base Station Sleeping for Energy Saving in Cellular Networks

Deep Reinforcement Learning for 5G Networks: Joint Beamforming, Power Control, and Interference Coordination

Reduction in Energy Consumption of the 5G Communication System and Beyond Through Collaborative Optimization for BS Site Operation: Challenges, Efforts and the New Approach