Abstract:This research is concerned with the novel application and investigation of `Soft Actor Critic' (SAC) based Deep Reinforcement Learning (DRL) to control the cooling setpoint (and hence cooling loads) of a large commercial building to harness energy flexibility. The research is motivated by the challenge associated with the development and application of conventional model-based control approaches at scale to the wider building stock. SAC is a model-free DRL technique that is able to handle continuous action spaces and which has seen limited application to real-life or high-fidelity simulation implementations in the context of automated and intelligent control of building energy systems. Such control techniques are seen as one possible solution to supporting the operation of a smart, sustainable and future electrical grid. This research tests the suitability of the SAC DRL technique through training and deployment of the agent on an EnergyPlus based environment of the office building. The SAC DRL was found to learn an optimal control policy that was able to minimise energy costs by 9.7% compared to the default rule-based control (RBC) scheme and was able to improve or maintain thermal comfort limits over a test period of one week. The algorithm was shown to be robust to the different hyperparameters and this optimal control policy was learnt through the use of a minimal state space consisting of readily available variables. The robustness of the algorithm was tested through investigation of the speed of learning and ability to deploy to different seasons and climates. It was found that the SAC DRL requires minimal training sample points and outperforms the RBC after three months of operation and also without disruption to thermal comfort during this period. The agent is transferable to other climates and seasons although further retraining or hyperparameter tuning is recommended.

Energy optimization for regional buildings based on distributed reinforcement learning

Operational Optimization for Off-Grid Renewable Building Energy System Using Deep Reinforcement Learning

Coordinated energy management for integrated energy system incorporating multiple flexibility measures of supply and demand sides: A deep reinforcement learning approach

Heating ventilation air‐conditioner system for multi‐regional commercial buildings based on deep reinforcement learning

Multi-agent reinforcement learning dealing with hybrid action spaces: A case study for off-grid oriented renewable building energy system

A Novel Operation Method for Renewable Building by Combining Distributed DC Energy System and Deep Reinforcement Learning

Energy Management of Intelligent Building Based on Deep Reinforced Learning

Energy-Efficient HVAC Control Based on Reinforcement Learning and Transfer Learning in a Residential Building

Study on the Application of Reinforcement Learning in the Operation Optimization of HVAC System

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning

Real building implementation of a deep reinforcement learning controller to enhance energy efficiency and indoor temperature control

Real-World Implementation of Reinforcement Learning Based Energy Coordination for a Cluster of Households

Energy Optimization in Sustainable Smart Environments With Machine Learning and Advanced Communications

Comparison of reinforcement learning and model predictive control for building energy system optimization

Reinforcement Learning for Optimal Control of a District Cooling Energy Plant

Comparative Analysis of Reinforcement Learning Approaches for Multi-Objective Optimization in Residential Hybrid Energy Systems

Development of a Soft Actor Critic Deep Reinforcement Learning Approach for Harnessing Energy Flexibility in a Large Office Building

Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning Control

Cooling Water System Control of a Commercial Complex Based on Reinforcement Learning

Optimizing Data Centre Energy Efficiency via Event-Driven Deep Reinforcement Learning