Abstract:With the application of renewable energy in single office buildings, an increasing number of power grids require building systems coupled with renewable energy to realize off-grid operation. However, the uncertainty of renewable energy sources and the safety of the corresponding energy storage equipment have become major challenges for these systems. Reinforcement learning has made considerable progress in the field of building control as an advanced control algorithm; however, research on its application to the off-grid operation of renewable energy systems, particularly the specific reward function design is limited. Therefore, this study considered the off-grid operation of a renewable building energy system and the safety (prevention of deterioration) of its battery as optimization goals. This study is based on a real building energy system, through the operation control of generators, solar photovoltaics, and batteries to achieve optimization purposes. Aiming at these two optimization goals, this study introduces a detailed reward function design and complete reinforcement learning workflow through a Gaussian distribution. Two deep reinforcement learning (DRL) algorithms were trained and verified by offline reinforcement learning based on the measured data-sets of actual existing buildings in Japan. The results show that the proposed reinforcement learning design can better achieve the two optimization goals of off-grid operation and battery safety under ordinary and extreme conditions. On off-grid operation tasks, the best DRL algorithm can achieve a mean hourly grid power purchase error of less than 2 kWh for the entire optimization window. Furthermore, the proposed deep reinforcement learning algorithms can simultaneously maintain a maximum average 7.72 h of battery unsafe state over the entire 168-hour optimization window.

Real-time optimization of large-scale hydrogen production systems using off-grid renewable energy: Scheduling strategy based on deep reinforcement learning

Learning-based scheduling of industrial hybrid renewable energy systems

Two-layered optimal scheduling under a semi-model architecture of hydro-wind-solar multi-energy systems with hydrogen storage

Deep Reinforcement Learning Based Energy Scheduling of a Hybrid Electricity/Heat/Hydrogen Energy System

Distributed Optimal Scheduling of Multi-Microgrid System with Hydrogen Based on Model Predictive Control

A Data‐driven Scheduling Approach for Integrated Electricity‐hydrogen System Based on Improved DDPG

Optimal Energy Scheduling for Microgrids with a Hybrid Battery-Hydrogen Storage System Based on Reinforcement Learning

Research on Energy Management in Hydrogen–Electric Coupled Microgrids Based on Deep Reinforcement Learning

Operational Optimization for Off-Grid Renewable Building Energy System Using Deep Reinforcement Learning

Optimal Scheduling in IoT-Driven Smart Isolated Microgrids Based on Deep Reinforcement Learning

Dynamic optimization of an integrated energy system with carbon capture and power-to-gas interconnection: A deep reinforcement learning-based scheduling strategy

A Deep-Reinforcement-Learning-based Optimal Scheduling for Rural Integrated Energy System Considering the Uncertainties of Renewable Energies and Loads

Deep Reinforcement Learning Based Energy Management of a Hybrid Electricity-Heat-Hydrogen Energy System with Demand Response

Real-Time Scheduling Method Based on Deep Reinforcement Learning for a Hybrid Wind-Solar-Storage Energy Generation System

Research on Energy Management of Hydrogen Electric Coupling System Based on Deep Reinforcement Learning

Intelligent Energy Scheduling in Renewable Integrated Microgrid with Bidirectional Electricity-to-Hydrogen Conversion

Real-Time Scheduling of High-Penetrated Renewable Power Systems: an Expert Knowledge and Reinforcement Learning Hybrid Approach

Forecast-driven stochastic optimization scheduling of an energy management system for an isolated hydrogen microgrid

Adaptively optimal energy management for integrated hydrogen energy systems

Physical model-assisted deep reinforcement learning for energy management optimization of industrial electric-hydrogen coupling system with hybrid energy storage

Power balance control of RES integrated power system by deep reinforcement learning with optimized utilization rate of renewable energy