Abstract:The electric power grid is changing from a traditional power system to a modern, smart, and integrated power system. Microgrids (MGs) play a vital role in combining distributed renewable energy resources (RESs) with traditional electric power systems. Intermittency, randomness, and volatility constitute the disadvantages of distributed RESs. MGs with high penetrations of renewable energy and random load demand cannot ignore these uncertainties, making it difficult to operate them effectively and economically. To realize the optimal scheduling of MGs, a real-time economic energy management strategy based on deep reinforcement learning (DRL) is proposed in this paper. Different from traditional model-based approaches, this strategy is learning based, and it has no requirements for an explicit model of uncertainty. Taking into account the uncertainties in RESs, load demand, and electricity prices, we formulate a Markov decision process for the real-time economic energy management problem of MGs. The objective is to minimize the daily operating cost of the system by scheduling controllable distributed generators and energy storage systems. In this paper, a deep deterministic policy gradient (DDPG) is introduced as a method for resolving the Markov decision process. The DDPG is a novel policy-based DRL approach with continuous state and action spaces. The DDPG is trained to learn the characteristics of uncertainties of the load, RES output, and electricity price using historical data from real power systems. The effectiveness of the proposed approach is validated through the designed simulation experiments. In the second experiment of our designed simulation, the proposed DRL method is compared to DQN, SAC, PPO, and MPC methods, and it is able to reduce the operating costs by 29.59%, 17.39%, 6.36%, and 9.55% on the June test set and 30.96%, 18.34%, 5.73%, and 10.16% on the November test set, respectively. The numerical results validate the practical value of the proposed DRL algorithm in addressing economic operation issues in MGs, as it demonstrates the algorithm's ability to effectively leverage the energy storage system to reduce the operating costs across a range of scenarios.

Reinforcement learning based two‐timescale energy management for energy hub

Safe Reinforcement Learning for Real-Time Automatic Control in a Smart Energy-Hub

A Multi-Agent Deep-Reinforcement-Learning-Based Strategy for Safe Distributed Energy Resource Scheduling in Energy Hubs

Two-timescale autonomous energy management strategy based on multi-agent deep reinforcement learning approach for residential multicarrier energy system

Multi-level coordinated energy management for energy hub in hybrid markets with distributionally robust scheduling

An Integrated Demand Response-Based Energy Management Strategy for Integrated Energy System Based on Deep Reinforcement Learning

Deep Reinforcement Learning for Smart Home Energy Management

Coordinated energy management for integrated energy system incorporating multiple flexibility measures of supply and demand sides: A deep reinforcement learning approach

Optimal energy management of energy hub: A reinforcement learning approach

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment

Safe reinforcement learning based optimal low-carbon scheduling strategy for multi-energy system

Data-driven approaches to achieve energy transition: Statistical analysis of demand-side management and smart decision making in urban energy hub networks

A Real-Time Rolling Horizon Chance Constrained Optimization Model for Energy Hub Scheduling

Distributed flexible resource regulation strategy for residential communities based on deep reinforcement learning

Battery and Hydrogen Energy Storage Control in a Smart Energy Network with Flexible Energy Demand Using Deep Reinforcement Learning

A Novel Model-Free Deep Reinforcement Learning Framework for Energy Management of a PV Integrated Energy Hub

Energy management based on multi-agent deep reinforcement learning for a multi-energy industrial park

Deep reinforcement learning for real-time economic energy management of microgrid system considering uncertainties

Real-time Energy Management of a Low-Carbon Micro-Grid Based on Deep Reinforcement Learning

An optimal solutions-guided deep reinforcement learning approach for online energy storage control

A Hybrid Data-Driven Method for Low-Carbon Economic Energy Management Strategy in Electricity-Gas Coupled Energy Systems Based on Transformer Network and Deep Reinforcement Learning