Abstract:With the application of renewable energy in single office buildings, an increasing number of power grids require building systems coupled with renewable energy to realize off-grid operation. However, the uncertainty of renewable energy sources and the safety of the corresponding energy storage equipment have become major challenges for these systems. Reinforcement learning has made considerable progress in the field of building control as an advanced control algorithm; however, research on its application to the off-grid operation of renewable energy systems, particularly the specific reward function design is limited. Therefore, this study considered the off-grid operation of a renewable building energy system and the safety (prevention of deterioration) of its battery as optimization goals. This study is based on a real building energy system, through the operation control of generators, solar photovoltaics, and batteries to achieve optimization purposes. Aiming at these two optimization goals, this study introduces a detailed reward function design and complete reinforcement learning workflow through a Gaussian distribution. Two deep reinforcement learning (DRL) algorithms were trained and verified by offline reinforcement learning based on the measured data-sets of actual existing buildings in Japan. The results show that the proposed reinforcement learning design can better achieve the two optimization goals of off-grid operation and battery safety under ordinary and extreme conditions. On off-grid operation tasks, the best DRL algorithm can achieve a mean hourly grid power purchase error of less than 2 kWh for the entire optimization window. Furthermore, the proposed deep reinforcement learning algorithms can simultaneously maintain a maximum average 7.72 h of battery unsafe state over the entire 168-hour optimization window.

Battery Management for Warehouse Robots Via Average-Reward Reinforcement Learning

Battery Management for Automated Warehouses via Deep Reinforcement Learning.

Deep Reinforcement Learning for Dynamic Scheduling of Energy-Efficient Automated Guided Vehicles

Real-Time Battery Thermal Management for Electric Vehicles Based on Deep Reinforcement Learning

Real-Time Charging Scheduling of Automated Guided Vehicles in Cyber-Physical Smart Factories Using Feature-Based Reinforcement Learning

Operational Optimization for Off-Grid Renewable Building Energy System Using Deep Reinforcement Learning

Multi-Agent Deep Reinforcement Learning for Recharging-Considered Vehicle Scheduling Problem in Container Terminals

Deep Reinforcement Learning for Dynamic Order Picking in Warehouse Operations

Multi-Objective Optimization of AGV Real-Time Scheduling Based on Deep Reinforcement Learning

Reward shaping to improve the performance of deep reinforcement learning in perishable inventory management

Battery Health-Aware and Deep Reinforcement Learning-Based Energy Management for Naturalistic Data-Driven Driving Scenarios

RTAW: An Attention Inspired Reinforcement Learning Method for Multi-Robot Task Allocation in Warehouse Environments

An Intelligent Energy Management Strategy for Hybrid Vehicle with Irrational Actions Using Twin Delayed Deep Deterministic Policy Gradient

Toward Energy-Efficient Routing of Multiple AGVs with Multi-Agent Reinforcement Learning

Enhancing Battery Storage Energy Arbitrage with Deep Reinforcement Learning and Time-Series Forecasting

Probabilistic Automata-Based Method for Enhancing Performance of Deep Reinforcement Learning Systems

Average-Reward Reinforcement Learning with Trust Region Methods

Can Deep Reinforcement Learning Improve Inventory Management? Performance on Dual Sourcing, Lost Sales and Multi-Echelon Problems

AGV path planning and optimization with deep reinforcement learning model

Study on an Average Reward Reinforcement Learning Algorithm

Intelligent Path Planning for AGV-UAV Transportation in 6G Smart Warehouse