Abstract:With the application of renewable energy in single office buildings, an increasing number of power grids require building systems coupled with renewable energy to realize off-grid operation. However, the uncertainty of renewable energy sources and the safety of the corresponding energy storage equipment have become major challenges for these systems. Reinforcement learning has made considerable progress in the field of building control as an advanced control algorithm; however, research on its application to the off-grid operation of renewable energy systems, particularly the specific reward function design is limited. Therefore, this study considered the off-grid operation of a renewable building energy system and the safety (prevention of deterioration) of its battery as optimization goals. This study is based on a real building energy system, through the operation control of generators, solar photovoltaics, and batteries to achieve optimization purposes. Aiming at these two optimization goals, this study introduces a detailed reward function design and complete reinforcement learning workflow through a Gaussian distribution. Two deep reinforcement learning (DRL) algorithms were trained and verified by offline reinforcement learning based on the measured data-sets of actual existing buildings in Japan. The results show that the proposed reinforcement learning design can better achieve the two optimization goals of off-grid operation and battery safety under ordinary and extreme conditions. On off-grid operation tasks, the best DRL algorithm can achieve a mean hourly grid power purchase error of less than 2 kWh for the entire optimization window. Furthermore, the proposed deep reinforcement learning algorithms can simultaneously maintain a maximum average 7.72 h of battery unsafe state over the entire 168-hour optimization window.

Optimal Operation Strategy for Distribution Grid with High Penetration Renewable Energy: a Novel Deep Reinforcement Learning Based Approach

DDPG-based Reactive Power Optimization Strategy for Active Distribution Network

Optimal Scheduling of a Wind Energy Dominated Distribution Network Via a Deep Reinforcement Learning Approach

Operational Optimization for Off-Grid Renewable Building Energy System Using Deep Reinforcement Learning

Research on Optimal Control Strategy of Distributed Photovoltaic Based on Deep Reinforcement Learning

Deep Reinforcement Learning Based Approach for Optimal Power Flow of Distribution Networks Embedded with Renewable Energy and Storage Devices

Double Deep Q-learning Based Real-Time Optimization Strategy for Microgrids

Optimizing Load Scheduling in Power Grids Using Reinforcement Learning and Markov Decision Processes

A Holistic Power Optimization Approach for Microgrid Control Based on Deep Reinforcement Learning

Real-time Operation Optimization in Active Distribution Networks Based on Multi-Agent Deep Reinforcement Learning

Reinforcement Learning and Stochastic Optimization with Deep Learning-Based Forecasting on Power Grid Scheduling

Real-time Energy Management of a Low-Carbon Micro-Grid Based on Deep Reinforcement Learning

Data-Driven Online Energy Scheduling of a Microgrid Based on Deep Reinforcement Learning

Deep Reinforcement Learning Based Optimal Scheduling of Active Distribution System Considering Distributed Generation, Energy Storage and Flexible Load

Deep Reinforcement Learning Based Approach for Dynamic Optimal Power Flow in Active Distribution Network

Operational Optimization for the Grid-Connected Residential Photovoltaic-Battery System Using Model-Based Reinforcement Learning

Deep Reinforcement Learning-Based Method for Joint Optimization of Mobile Energy Storage Systems and Power Grid with High Renewable Energy Sources

A Deep Reinforcement Learning Based Approach for Optimal Active Power Dispatch

Double Deep Q-learning Based Approach for Power Flow Optimization in Distribution Networks

Day-Ahead Optimal Dispatch in Active Distribution Network Based on Deep Reinforcement Learning with Improved Feature Extraction Network

Deep Reinforcement Learning for Optimal Power Flow with Renewables Using Graph Information