Abstract:With the application of renewable energy in single office buildings, an increasing number of power grids require building systems coupled with renewable energy to realize off-grid operation. However, the uncertainty of renewable energy sources and the safety of the corresponding energy storage equipment have become major challenges for these systems. Reinforcement learning has made considerable progress in the field of building control as an advanced control algorithm; however, research on its application to the off-grid operation of renewable energy systems, particularly the specific reward function design is limited. Therefore, this study considered the off-grid operation of a renewable building energy system and the safety (prevention of deterioration) of its battery as optimization goals. This study is based on a real building energy system, through the operation control of generators, solar photovoltaics, and batteries to achieve optimization purposes. Aiming at these two optimization goals, this study introduces a detailed reward function design and complete reinforcement learning workflow through a Gaussian distribution. Two deep reinforcement learning (DRL) algorithms were trained and verified by offline reinforcement learning based on the measured data-sets of actual existing buildings in Japan. The results show that the proposed reinforcement learning design can better achieve the two optimization goals of off-grid operation and battery safety under ordinary and extreme conditions. On off-grid operation tasks, the best DRL algorithm can achieve a mean hourly grid power purchase error of less than 2 kWh for the entire optimization window. Furthermore, the proposed deep reinforcement learning algorithms can simultaneously maintain a maximum average 7.72 h of battery unsafe state over the entire 168-hour optimization window.

Data-Driven Wind Farm Volt/VAR Control Based on Deep Reinforcement Learning

A deep reinforcement learning method for managing wind farm uncertainties through energy storage system control and external reserve purchasing

Intelligent Wind Farm Control via Grouping-Based Reinforcement Learning

Two-Stage Deep Reinforcement Learning for Inverter-Based Volt-VAR Control in Active Distribution Networks

Reactive-Voltage Coordinated Control of Offshore Wind Farm Based on Deep Reinforcement Learning

Data-Driven Robust Voltage/VAR Control Using PV Inverters in Active Distribution Networks

Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems

Data-Driven Volt/VAR Optimization for Modern Distribution Networks: A Review

Operational Optimization for Off-Grid Renewable Building Energy System Using Deep Reinforcement Learning

Deep Reinforcement Learning Strategy for Electric Vehicle Charging Considering Wind Power Fluctuation

Deep Reinforcement Learning for Voltage Control and Renewable Accommodation Using Spatial-Temporal Graph Information

Efficient Learning of Voltage Control Strategies via Model-based Deep Reinforcement Learning

Robust Federated Deep Reinforcement Learning for Optimal Control in Multiple Virtual Power Plants with Electric Vehicles

Wind Farm Power Generation Control Via Double-Network-Based Deep Reinforcement Learning

Review of Deep Reinforcement Learning and Its Application in Modern Renewable Power System Control

Robust Offline Deep Reinforcement Learning for Volt-Var Control in Active Distribution Networks

A Deep Reinforcement Learning Based Approach for Optimal Active Power Dispatch

Deep Reinforcement Learning Based Wind Farm Cluster Reactive Power Optimization Considering Network Switching

Reinforcement Learning for Volt-Var Control: A Novel Two-stage Progressive Training Strategy

Reinforcement Learning for Fluctuation Reduction of Wind Power with Energy Storage